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Executive  Summary 

After  Hurricane  Katrina  struck  New  Orleans  on  August  29,  2005,  the  U.S.  Army  Corps  of 
Engineers  (US ACE)  immediately  started  rebuilding  the  canal  walls  along  the  three  main 
drainage  canals  and  implementing  additional  flood  protection  for  the  City  of  New 
Orleans.  The  mission  was  to  complete  these  additional  flood  protection  features  by  June 
2006  in  anticipation  of  the  2006  hurricane  season.  USACE  installed  three  temporary 
pumping  structures  with  floodgates  at  the  outfall  of  the  three  main  drainage  canals  to 
Lake  Pontchartrain.  The  gates  were  designed  to  remain  open  under  normal  operating 
conditions  and  be  closed  during  a  storm  event  when  storm  surge  from  Lake  Pontchartrain 
might  exceed  safe  levels.  With  the  gates  shut,  the  storm  surge  from  Lake  Pontchartrain 
would  be  prevented  from  entering  the  drainage  canals,  though  the  pump  systems  would 
still  allow  for  the  discharge  of  stormwater  being  pumped  by  the  New  Orleans  Sewerage 
and  Water  Board  (SWB).  A  total  of  40  large  capacity  hydraulic  pumps  were  installed  by 
USACE  in  2006  to  assist  with  the  removal  of  stormwater  from  the  three  main  drainage 
canals  into  the  lake  when  the  gates  were  closed.  Additionally,  19  direct  drive  pumps 
were  installed  in  2007.  This  ensured  the  canal  water  elevations  were  kept  at  “safe”  levels 
as  established  by  USACE  geotechnical  engineers.  Approximately  16,600  cubic  feet  per 
second  (cfs)  (7.5  million  gallons  per  minute  [gpm])  in  pumping  capacity  was  designed, 
procured,  constructed,  and  tested  in  approximately  2 1  months. 

Moving  Water  Industries  (MWI)  provided  the  major  equipment  for  the  hydraulic 
pumping  systems.  A  USACE  inspection  team  was  dispatched  to  MWI  to  observe  and 
record  the  equipment  assembly  and  testing  at  the  factory  prior  to  shipment  in  order  to 
facilitate  the  objectives  within  the  timeframe  allowed.  During  this  period  of  observations, 
questions  were  raised  by  a  whistleblower  relative  to  observed  results  and  testing  protocol 
for  some  of  the  various  startup  and  performance  tests  of  the  MWI  hydraulic  pumping  unit. 
These  questions  led  to  formal  investigations  by  USACE,  the  Department  of  Defense 
Inspector  General’s  (DoDIG’s)  Office,  and  the  U.S.  Government  Accounting  Office 
(GAO),  each  of  which  issued  reports  on  their  findings.  Subsequent  to  the  release  of  those 
reports,  questions  continued  to  be  raised  by  the  Office  of  Special  Counsel  (OSC) 
regarding  the  adequacy  of  the  pump  equipment  testing  and  vulnerability  to  failure  in  the 
event  of  a  hurricane;  therefore  the  OSC  recommended  that  a  thorough  and  impartial 
investigation  be  conducted  by  independent  professional  engineers. 

In  response  to  a  request  for  engineering  services  by  the  DoDIG,  Parsons  assembled  a 
team  of  professional  engineers  experienced  in  pump  station  design,  operations, 
maintenance,  and  testing  requirements  This  team’s  objective  was  to  provide  an 
independent  engineering  assessment  of  the  pump  testing  adequacy  and  to  identify 
potential  vulnerabilities  of  the  temporary  hydraulic  pumping  systems  at  17th  Street  Canal, 
London  Avenue  Canal,  and  Orleans  Avenue  Canal.  In  order  to  address  these  two  aspects 
of  the  temporary  pump  stations’  readiness,  Parsons  assembled  two  separate  but 
collaborative  teams.  One  team  was  assembled  for  the  testing  adequacy  evaluation  and  the 
other  team  was  assembled  for  the  vulnerability  analysis. 

The  testing  adequacy  analysis  was  conducted  through  a  progression  of  intelligence 
gathering.  This  effort  started  with  a  review  of  existing  test  data  and  reports  that  had  been 
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developed  to  date.  After  reviewing  the  available  data,  the  team  conducted  site  visits  at  the 
New  Orleans  temporary  pump  stations  at  17th  Street  Canal,  London  Avenue  Canal,  and 
Orleans  Avenue  Canal.  During  this  site  visit,  interviews  with  US  ACE  project  officials 
and  operations  staff  at  the  Hurricane  Protection  Office  (HPO)  and  the  New  Orleans  SWB 
Director  were  conducted.  While  in  New  Orleans,  the  team  was  presented  with  the  testing 
methodology  conducted  by  the  USACE  Engineer  Research  and  Development  Center 
(ERDC).  Details  of  the  assumptions  and  protocols  employed  by  ERDC  during  the  model 
testing  were  discussed  comprehensively  with  ERDC.  Following  the  site  visit  to  New 
Orleans,  the  team  conducted  interviews  with  individuals  from  the  pump  manufacturer, 
MWI  in  Deerfield  Beach,  Florida.  The  Parsons  team  toured  the  facilities  to  observe  the 
testing  configurations  and  fabrication  methods  employed  by  MWI.  Additional 
engineering  data  was  gathered  from  MWI  to  facilitate  assessment  of  the  factory  tests. 
Once  the  MWI  factory  visit  was  completed,  the  USACE  Jacksonville  District  Quality 
Assurance  (QA)  team  was  interviewed  to  obtain  their  first  hand  observations  during  the 
hydraulic  pump  fabrication  and  testing.  The  QA  team  substantially  confirmed  the 
observations  documented  in  the  DoDIG  and  GAO  reports.  Finally,  the  Parsons  team  met 
with  the  whistleblower  at  the  local  USACE  Palm  Beach  Gardens  office  to  obtain  first 
hand  information  on  the  testing  of  the  pumps.  The  team  was  presented  with  a  detailed 
account  on  the  testing  and  observation  procedures. 

The  vulnerability  progression  was  similar  to  the  testing  adequacy  approach  including  all 
of  the  site  visits  and  interviews  discussed  above.  In  addition,  this  intelligence  gathering 
effort  included  a  supplemental  trip  to  New  Orleans  to  discuss  the  sequence  of  events 
associated  with  the  acceptance  testing  and  a  more  in-depth  discussion  regarding  the 
maintenance  program.  The  vulnerability  analysis  inherently  involves  the  judgment  of 
professional  engineers  using  the  data  provided,  supplemented  by  experience  to  render  an 
opinion  of  system  vulnerability.  This  effort  is  detailed  in  Section  3  of  this  report  which 
describes  the  systematic  process  of  identification  and  categorization  of  vulnerabilities. 
The  areas  studied  were  mechanical,  electrical,  structural,  operational,  and  the 
maintenance  program. 

After  the  aforementioned  data  was  analyzed  and  the  conclusions  documented  the  Parsons 
team  reviewed  perfonnance  data  obtained  during  Hurricanes  Gustav  and  Ike.  This 
approach  helped  the  team  maintain  maximum  objectivity  in  analyzing  pre-Gustav  records. 

The  information  reviewed  in  preparing  this  report  included  prior  reports,  data  previously 
gathered  by  the  DoDIG  including  documentation  submitted  by  a  whistleblower  to  the  U.S. 
Office  of  Special  Counsel,  information  Parsons  requested  during  the  course  of  the 
assessment,  and  interviews  Parsons  requested  in  their  assessment  plan. 

The  findings  and  conclusions  of  the  Parsons  team  are  as  follows: 

1.  Based  on  the  infonnation  provided  and  the  interviews  performed,  the  Parsons 
team  found  that  there  were  issues  with  the  factory  testing  and  changes  to  testing 
procedures  by  USACE  that  took  place  during  the  testing  process.  Further 
investigations  also  show  issues  raised  by  the  whistleblower  have  been  rectified  in 
the  field  and  the  pumps  re-tested  for  full  functionality.  Therefore,  it  is  the  Parsons 
team’s  opinion  that  the  hydraulic  pump  systems  have  been  adequately  tested  for 
their  intended  purpose.  Contract  administration  and  compliance  issues  were 
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addressed  by  others  under  previously  issued  reports  and  are  not  part  of  the  scope 
for  this  effort. 

2.  The  second  part  of  the  report  deals  with  identifying  potential  vulnerabilities  to 
failure  of  the  40  hydraulic  pumping  systems  installed.  It  is  the  opinion  of  the 
Parsons  team  that  given  the  project  objectives,  the  selection  of  hydraulic  pumping 
systems  was  appropriate.  These  systems  are  designed  for  use  where  rapid 
deployment  is  desired  and  the  need  to  accommodate  flexible  site  configurations  is 
necessary.  While  hydraulic  pumping  systems  inherently  require  more 
maintenance  than  other  less  flexible  and  longer  lead  time  pumping  systems,  these 
hydraulic  systems  exhibit  no  higher  level  of  vulnerability  than  other  similarly 
installed  systems  with  similar  complexity,  as  long  as  the  recommended  inspection 
and  maintenance  activities  occur.  The  temporary  nature  of  this  installation  also 
affects  the  vulnerability  analysis.  For  example,  corrosion  protection  measures  for 
a  temporary  facility  are  different  than  those  for  a  pennanent  installation,  since  the 
design  life  of  the  temporary  structure  is  much  less.  It  is  the  opinion  of  the  Parsons 
team  that  as  long  as  the  permanent  facilities  proceed  according  to  schedule  and  a 
thorough  inspection  and  maintenance  program  is  followed  for  the  temporary 
facilities,  there  are  no  immediate  vulnerabilities  to  catastrophic  failures  with  the 
hydraulic  pumping  systems  or  their  supporting  systems. 

3.  On  August  31,  2008,  Hurricane  Gustav  made  land  fall,  with  a  hurricane  force  of  a 
Category  2  stonn,  the  Louisiana  coast  experienced  torrential  rain  and  high  winds 
of  approximately  100  miles  per  hour  (mph),  which  generated  a  storm  surge  in 
Lake  Pontchartrain  of  4.8  feet.  Records  show  that  the  USACE  canal  teams 
received  orders  to  close  the  canal  gates  at  the  temporary  outflow  canal  pump 
stations  at  the  17th  Street  and  London  Avenue  canals,  cutting  off  the  canals 
outflow  to  Lake  Pontchartrain  in  anticipation  of  the  storm  surge  associated  with 
the  high  winds.  Pumps  were  put  into  service  and  the  two  canals  were  successfully 
kept  at  the  safe  water  levels.  The  Orleans  Avenue  gates  were  not  shut  as  the  water 
levels  were  at  a  safe  level. 

On  the  morning  of  September  12,  2008,  Hurricane  Ike  made  land  fall  as  a 
Category  3  wind  force  generating  a  storm  surge  in  Lake  Pontchartrain  of 
approximately  5.2  feet.  The  coast  experienced  rain  and  winds  of  around  25  mph 
and  at  the  temporary  pump  stations,  the  USACE  canal  team  received  orders  to 
close  the  canal  gates  cutting  off  the  canals’  outflow  to  Lake  Pontchartrain.  Again 
pumps  were  put  into  service  and  the  two  canals  were  successfully  kept  at  the  safe 
water  levels.  The  Orleans  Avenue  gates  were  not  shut  as  the  water  levels  were 
below  its  designated  safe  level. 

It  is  the  opinion  of  the  Parsons  team  that  the  temporary  hydraulic  pumping 
systems  performed  successfully,  keeping  the  water  levels  of  the  canals  at  the 
determined  safe  level  for  both  hurricanes. 


_ Final  Report _ 

PARSONS  ES  -  3  February  27,  2009 


Independent  Engineering  Assessment  of  the 
New  Orleans  Temporary  Outfall  Canal  Pumps 


Introduction 


Section  1  -  Introduction 

1.1  Background 

Rainwater  and  other  sources  of  stonnwater  from  the  City  of  New  Orleans  are  collected 
via  a  system  of  interior  canals  and  pumped  by  the  New  Orleans  SWB  into  three  main 
drainage  canals  located  at  17th  Street,  London  Avenue,  and  Orleans  Avenue.  These  three 
canals  historically  discharged  water  by  gravity  feed  to  Lake  Pontchartrain  as  part  of  the 
local  flood  control  system. 

On  August  29,  2005,  Hurricane  Katrina  blew  across  the  City  of  New  Orleans  with  winds 
in  the  Category  3  range  of  127  mph.  Tidal  surge  caused  Lake  Pontchartrain  to  rise  to  a 
level  that  not  only  prevented  gravity  discharge  but  also  caused  the  levee  protection 
system  along  the  outfall  canals  to  fail  resulting  in  the  well  documented  catastrophic 
flooding  of  New  Orleans  in  2005.  In  an  effort  to  mitigate  future  occurrences  similar  to 
that  experienced  in  Katrina,  US  ACE  was  authorized  to  design  and  construct  a  protection 
system  along  each  of  the  three  main  drainage  canals.  The  protection  system  includes 
reinforcement  of  the  levees,  construction  of  an  operable  gate  structure  (flood  gates)  to 
protect  against  storm  surge  and  the  installation  of  temporary  pumping  systems. 

USAGE  procured  and  installed  40  large  capacity  hydraulic  pumping  systems  to  provide 
pumping  capacity  at  the  17th  Street  Canal,  London  Avenue  Canal,  and  Orleans  Avenue 
Canal.  Each  hydraulic  pumping  system  consists  of  the  pumping  unit,  hydraulic  oil 
supply  and  return  lines,  and  water  discharge  piping.  The  60-inch  diameter  water  pump 
unit  contains  a  Hydraulic  Power  Unit  (HPU)  with  a  diesel  engine  and  hydraulic  pump. 
The  supporting  systems  include  the  structural  support  system,  electronic  data  and 
communication  system,  the  electrical  support  system,  and  the  mechanical  support  system. 

Permanent  pump  stations  are  scheduled  to  be  constructed  by  2013.  When  the  newly 
constructed  flood  gates  are  closed,  blocking  the  normally  occurring  gravity  flow,  the 
temporary  pump  systems  pump  the  collected  stormwater  from  the  drainage  canals  to 
Lake  Pontchartrain.  Approximately  16,600  cubic  feet  per  second  (cfs)  (7.5  million  gpm) 
in  pumping  capacity  was  designed,  procured,  constructed,  and  tested  in  approximately  2 1 
months.  Table  1-1  below  is  a  summary  of  the  general  configuration  of  the  three 
temporary  pump  stations: 


Table  1-1 — Summary  of  the  General  Configuration  of  the  Three  Pump  Stations 


Pump  Station 

Hydraulic  Pump 

Direct  Drive  Pump 

Portable  Pump 

17th  Street  Canal 

18 

11 

20 

London  Ave.  Canal 

12 

8 

- 

Orleans  Ave.  Canal 

10 

- 

- 

MWI  was  selected  by  US  ACE  to  provide  the  major  equipment  for  the  hydraulic  pumping 
systems.  In  order  to  facilitate  the  objectives  within  the  timeframe  allowed,  US  ACE 
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decided  to  send  an  inspection  team  to  MWI,  to  observe  and  record  the  equipment 
assembly  and  testing  that  occurred  at  the  factory  prior  to  shipment  of  the  equipment  to 
the  sites  in  New  Orleans.  During  this  period  of  observations  at  the  pump  manufacturer, 
questions  were  raised  relative  to  observed  results  and  testing  protocol  for  some  of  the 
various  startup  and  performance  tests  of  the  hydraulic  pumping  systems  provided  by 
MWI.  These  questions,  raised  by  a  whistleblower,  led  to  a  formal  investigation  by 
USACE,  the  DoDIG’s  Office,  and  the  GAO,  each  of  which  issued  reports  on  their 
findings.  Subsequent  to  the  release  of  those  reports,  questions  continued  to  be  raised 
regarding  the  pumping  equipment. 

1.2  Scope 

Parsons  was  retained  in  response  to  the  questions  raised  subsequent  to  the 
aforementioned  reports  and  the  recommendation  by  the  OSC  that  a  thorough  and 
impartial  investigation  be  conducted  by  independent  professional  engineers.  The  main 
objective  was  to  obtain  an  outside  opinion  from  a  professional  engineering  company  to 
resolve  lingering  and  additional  questions  regarding  the  capability  of  the  pumping 
systems. 

The  overall  objectives  of  the  assessment  are  to  review  the  adequacy  of  testing  of  the 
temporary  pumping  systems  and  to  identify  and  assess  vulnerabilities  of  the  hydraulic 
pumping  systems  to  failures  in  the  event  of  a  hurricane  (specifically  a  10-year,  24-hour 
rainfall  event  to  which  USACE  designed  the  systems).  This  assessment  does  not  include 
the  direct-drive,  diesel-driven  pumping  systems  supplied  by  others,  the  portable  pumps  at 
17th  Street  temporary  pump  station,  or  the  floodwall  and  levee  reinforcement  protection. 

1.3  Standards  and  References 

Industry  practice  for  the  design  and  testing  of  large  flood  control  pumps  and  pumping 
stations  typically  follows  the  Hydraulic  Institute  (HI)  standards  for  various  types  of 
pumps.  These  standards  are  printed  by  the  American  National  Standards  Institute  (ANSI) 
and  are  reviewed  on  a  minimum  5-year  cycle  as  required  by  changes  in  technology  and 
updates,  and  by  canvassing  of  engineers,  equipment  manufacturers,  and  end-users.  The 
Pump  Intake  Design  Standard  (ANSI/HI  9.8,  1998)  also  addresses  when  the  acceptance 
criteria  for  physically  modeling  the  pump  station  sump  and  approach  canal  can  be  relaxed. 

MWI  staff  advised  the  Parsons  team  that  the  codes  and  standards  used  by  their  company 
for  the  design,  manufacture,  and  assembly  of  their  pumping  systems  are  the  ANSI/HI 
standards  for  vertical  pumps,  the  American  Society  of  Mechanical  Engineers  (ASME) 
Perfonnance  Testing  Code  (PTC)  8.2,  Centrifugal  Pumps  and  the  ASME  B31  set  of 
process  piping  standards. 

The  types  of  pumping  systems  used  at  the  temporary  pump  stations  are  unique  in  the 
sense  that  there  is  not  a  specific  HI  standard,  which  covers  this  patented,  hybrid 
submersible  axial,  and  mixed-flow  propeller  pump  configuration.  The  Parsons  team 
concluded  that  the  most  applicable  classification  is  the  vertical  pump  classification  as 
described  by  HI.  Applicable  ANSI/HI  standards  for  the  design  and  testing  of  the  water 
pumps  include,  but  are  not  limited  to  the  following  list  of  2002  editions: 
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2.1  -  2.2  Vertical  Pumps  for  Nomenclature  and  Definitions 

2.3  Vertical  Pumps  for  Design  and  Application 

2.4  Vertical  Pumps  for  Installation,  Operation  and  Maintenance 

2.6  Vertical  Pump  Tests 

9.8  Centrifugal  and  Vertical  Pump  Intake  Design 

Other  applicable  standards  used  in  this  report  are: 

National  Electric  Code  (NEC)  -  2008  Edition 
ASME  Code  B31  for  Standard  of  Pressure  Piping 
ASME  Code  31.3  for  Standard  for  Process  Piping 

American  Society  of  Civil  Engineers  (ASCE)  7-02  Minimum  Design  Loads  for 
Building  and  Other  Structures  (Applicable  at  the  time  of  design) 
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Section  2  -  Testing  Adequacy  Analysis 
2.1  Pump  Unit  Description 

Each  hydraulic  pump  unit  consists  of  two  main  components.  The  first  component  is  a 
drive  unit  that  consists  of  a  prime  mover  (a  Caterpillar  diesel  engine),  which  drives  a 
double  vane  hydraulic  pump  (made  by  Denison  )  rated  for  3,200  pounds  per  square  inch 
(psi)  complete  with  an  oil  reservoir.  This  whole  first  component  is  labeled  the  HPU  for 
the  purposes  of  this  report.  (Refer  to  Figure  2-1.)  The  second  component,  the  MWI  water 
pump,  consists  of  an  encased  impeller,  which  is  driven  by  a  Rineer  hydraulic  motor.  This 
hydraulic  motor,  which  is  surrounded  by  the  pumped  water,  turns  a  short  vertical  shaft 
connected  to  the  stainless  steel  axial-flow  impeller  and  is  connected  to  the  HPU  by  a  twin 
set  of  hydraulic  hoses  and  pipes. 

The  temporary  pumping  facilities  at  each  canal  are  similar  in  configuration,  varying  in 
total  capacity  from  9200  cfs  at  17th  Street,  5200  cfs  at  London  Avenue  to  2200  cfs  at 
Orleans  Avenue.  There  are  a  total  of  40  of  the  large  hydraulic  pump  units  distributed 
among  the  three  temporary  pumping  facilities. 


Figure  2-1 — Pump  Unit  Diagram 

2.2  Testing  Summary 

The  tests  performed  by  MWI,  US  ACE,  and  ERDC  on  the  HPU  pump  equipment  can  be 
divided  into  three  different  categories — factory,  field,  and  laboratory.  A  time  line  of 
these  activities  is  shown  in  Figure  2-2. 

The  factory  testing  included  testing  of  each  of  the  individual  HPUs,  performance  testing 
of  the  water  pumps  using  both  a  scaled  model  and  full-sized  prototype  pump,  and 
pressure  testing  of  the  full-sized  prototype  pumps. 
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Figure  2-2 — Testing  Timeline 


The  factory  testing  took  place  at  the  manufacturer’s  facilities  in  Deerfield  Beach,  Florida 
and  Sebastian,  Florida.  The  field  testing  was  performed  as  the  pumps  were  installed  at 
the  three  temporary  pump  stations  in  New  Orleans  and  consisted  of  performance, 
acceptance,  and  endurance  testing.  The  laboratory  testing  performed  by  ERDC  involved 
scaled  physical  models  of  the  approach  channels  and  pump  sump  areas  that  were  used  to 
characterize  the  approach  flow  hydraulics  and  the  hydraulics  of  the  intakes  to  the  pumps. 
This  testing  took  place  at  the  ERDC  facilities  in  Vicksburg,  Mississippi.  It  should  be 
noted  that  the  factory,  field,  and  laboratory  testing  did  not  occur  sequentially,  but  they 
were  performed  concurrently  as  the  equipment  was  being  manufactured  and  the  stations 
were  being  designed  and  constructed. 

The  Parsons  team  reviewed  the  pump  test  reports  provided  and  found  the  reports  to  be 
complete  and  detailed,  containing  comprehensive  tables  and  figures. 

2.3  Interview  Summary 

Review  of  the  documentation  helped  the  team  develop  questions  for  onsite  interviews 
with  US  ACE  and  MWI  personnel.  The  interviews  and  site  visits  were  conducted  by 
Parsons  with  personnel  from  the  DoDIG  office  present  and  were  organized  with  the 
objectives  of  understanding  the  chronology  and  extent  of  testing  perfonned,  from  the 
initial  factory  testing  through  field  acceptance  testing. 

The  Parsons  team  visited  the  USACE  Hurricane  Protection  Office  (HPO)  in  New  Orleans 
from  November  4  to  November  7,  2008.  During  those  meetings,  details  of  the  design  of 
the  pumping  facilities,  the  laboratory  testing  performed  at  ERDC,  and  the  testing 
requirements  (which  were  to  be  fulfilled  by  the  manufacturer  were  discussed).  The  three 
temporary  pump  facilities  were  visited  and  the  team  witnessed  startup  and  approximately 
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one  hour  of  pumping  at  the  17th  Street  station  where  all  but  one  of  the  large  hydraulic 
pumps  were  operating  (one  pump  was  out  of  service  for  maintenance). 

A  visit  to  the  pump  manufacturer,  MWI,  took  place  November  18,  2008.  During  that 
meeting,  the  details  of  the  factory  testing  were  discussed  and  a  tour  of  the  manufacturer’s 
Deerfield  Beach,  Florida,  test  and  assembly  facilities  followed.  On  November  20,  2008, 
the  Parsons  team  met  with  members  of  the  US  ACE  inspection  team  who  were  onsite  at 
the  pump  manufacturer’s  site  during  the  factory  testing  of  the  equipment. 

On  January  28,  2009,  the  Parsons  team  met  with  the  whistleblower  at  the  Palm  Beach 
Gardens  USAGE  office  and  conducted  an  interview.  During  the  meeting,  information  on 
the  extent  of  tests  carried  out  was  discussed  together  with  discussions  on  pump 
component  perfonnance.  Information  on  factory  test  methods,  outcome  of  factory  tests, 
and  site  tests  were  gathered  and  recorded  at  the  meeting. 

The  following  subsections  summarize  the  findings  of  the  documentation  review, 
interviews,  and  site  visits. 

2.4  Factory  Testing 

Details  of  the  HPU  testing,  scaled  model,  and  pump  prototype  testing  perfonned  at  the 
manufacturer’s  facilities  in  Deerfield  Beach  and  Sebastian  were  obtained  from  the  reports 
provided  by  the  DoDIG  and  interviews  conducted  with  MWI  personnel,  the  USAGE 
HPO  in  New  Orleans,  USAGE  inspectors  from  the  Jacksonville  District  and  the 
whistleblower. 

Typically,  these  types  of  tests  are  not  witnessed  by  the  purchaser  as  they  are  identified 
and  resolved  by  the  fabricator  before  the  inspection  effort.  Furthermore,  witnessed 
events  are  typically  limited  to  the  startup  and  commissioning  of  a  pump  station  except  in 
cases  where  the  specifications  require  witnessed  pump  and  driver  testing  events. 

2.4.1  Factory  Testing  of  the  Hydraulic  Pumping  Units 

The  equipment  specifications  called  for  the  HPUs  to  be  pressure  tested,  both  statically 
and  dynamically,  at  the  factory.  This  testing  would  provide  assurances  the  units  would 
be  operable  when  they  were  installed  at  the  site  and  minimize  the  commissioning  time 
normally  experienced  for  similar  equipment.  The  testing  program  originally  called  for 
each  unit  to  be  tested  statically  for  90  minutes  at  design  pressure  and  dynamically  for  1 5 
minutes  at  maximum  speed,  pressure,  and  temperature. 

In  early  factory  tests,  the  original  cams  (66&42)  of  the  Denison  pumps  on  the  HPUs  were 
found  to  be  underperforming.  These  were  replaced  with  66&50  cams  but  the  performance 
requirements  were  still  not  met  due  to  their  lack  of  ability  to  continuously  operate  at  the 
design  pressure  of  3,200  psi.  Dennison  replaced  all  the  cams  in  the  HPUs  with  72&45 
cams  at  no  cost  to  the  government.  The  available  data  indicates  that  all  HPUs  onsite 
today  have  the  72&45  cams  installed  and  have  proven  to  operate  successfully.  MWI 
recommended  operating  the  pumps  with  a  design  pressure  of  3,200  psi. 

As  documented  in  other  reports,  some  component  failures  occurred  during  the  factory 
tests.  The  subject  components  were  repaired  or  replaced  and  tests  resumed. 
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Recognizing  the  critical  schedule  constraints  to  have  the  pumps  on  site,  ready  to  operate 
in  the  hurricane  season,  delivery  schedules  and  tests  were  modified  during  the  course  of 
the  testing.  Test  requirements  were  modified  to  drop  initial  performance  testing  of  each 
of  the  water  pumps  and  test  all  the  HPUs  for  a  minimum  of  3  hours  at  lull  speed,  fully 
loaded.  Records  that  track  each  equipment  number  through  the  testing  process  show  all 
40  HPUs  went  through  a  3-hour  acceptance  test  at  the  factory.  Each  unit  under  test  was 
connected  to  a  hydraulic  water  pump  in  a  water  tank  and  tested  for  its  maximum  driving 
pressure.  One  HPU  experienced  engine  abnormalities  and  did  not  pass  the  3-hour 
acceptance  test.  This  unit  was  however  shipped  to  the  field  without  the  Government’s 
approval  of  testing.  USAGE  made  a  decision  to  allow  corrections  to  be  made  to  the  unit 
in  the  field  rather  than  sending  it  back  to  the  factory.  This  unit  passed  the  field  acceptance 
test  and  logged  25  running  hours. 

2.4.1.1  Findings 

While  there  are  standards  related  to  the  hydraulic  performance  of  pumps  of  this  type, 
there  is  no  industry  standard  for  the  factory  testing  of  the  drive  units  and  pumps  during 
the  manufacturing  process.  The  testing  performed  early  in  the  manufacturing  process  of 
the  HPUs  proved  beneficial  in  that  it  identified  assembly  and  performance  abnormalities 
in  the  drive  units  that  significantly  reduced  the  time  spent  on  acceptance  and 
commissioning  after  the  pumps  were  installed  onsite.  Abnormalities  encountered  with 
the  HPU  configuration  were  identified  and  corrected,  resulting  in  more  reliable  pieces  of 
equipment.  The  abnormalities  encountered  during  the  performance  tests  of  the  HPUs 
were  addressed.  Recognizing  critical  test  items  and  delivery  dates  were  at  risk,  the 
USAGE  officials  modified  the  test  procedures  in  the  course  of  production  but  did 
additional  field  testing  to  ensure  pump  operation  and  endurance. 

2.4.2  Factory  Prototype  Performance  Test 

A  factory  prototype  test  was  also  perfonned  (November  2006)  to  measure  the 
performance  of  a  full-scale  prototype  pump  and  compare  it  to  the  design  requirements. 
This  perfonnance  test  was  performed  and  observed  by  USACE.  The  report,  which  was 
reviewed  for  this  evaluation,  is  titled  “Data  Report  on  Factory  Tests  of  Discharge  and 
Total  Dynamic  Head  of  MWI  Pumps  used  on  New  Orleans  Outfall  Canals,”  December 
2006. 

2.4.2. 1  Flow  Measurements 

The  ERDC  report  indicates  numerous  difficulties  in  getting  consistent  readings  with  the 
test  setup  to  measure  flow.  Due  diligence  was  exercised  by  the  investigator  to  minimize 
the  error  related  to  velocity  differences  in  the  discharge  piping  at  the  point  of  flow 
measurement  by  taking  extensive  cross-sectional  measurements  and  using  accepted 
standards  to  correct  the  data  (United  States  Bureau  of  Reclamation  [USBR]  Water 
Measurement  Manual).  The  flow  measurement  points  are  shown  in  Figure  2-3.  The 
collected  data  was  corrected  using  standard  procedures  to  account  for  the  differences  in 
test  operating  speed  so  comparisons  could  be  made  with  the  contract  requirements  and  a 
previous  test  noted  as  “Measured  MWI  Analysis-288  rpm.” 
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Figure  2-3 — Flow  Measurement  Points 

USBR  10-pt  Velocity  Traverse.  D  =  pipe  diameter.  The  dimensions  are 
extracted  from  the  “Data  Report  on  Factory  Tests  of  Discharge  and  Total 
Dynamic  Head  of  MWI  Pumps  Used  on  New  Orleans  Outfall  Canals.” 

The  prototype  test  data  collected  was  then  plotted  on  the  curve  of  the  previously  collected 
MWI  prototype  data,  as  shown  in  Figure  2-4,  with  a  difference  found  in  the  slope  of  the 
curves.  The  Measured  ERDC  Analysis  -  288  revolutions  per  minute  (rpm)  test  curve  is 
parallel  but  lower  than  the  Measured  -  MWI  Analysis  -  288  rpm,  suggesting  there  may 
be  a  difference  in  the  method  of  velocity  calculations. 


Discharge,  GPM 


Figure  2-4 — Head-Discharge  Curves  Design 
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Figure  2-4  shows  head-discharge  curves  design,  measured  with  ERDC  analysis  and  with 
MWI  analysis,  the  discharge  is  based  on  the  average  of  all  4  traverses. 

2.4.2. 2  Findings 

The  size  of  these  pumps  necessitated  a  very  large  quantity  of  water  in  order  to  run  a  full 
test.  The  water  was  pumped  from  a  sump  and  was  continuously  recirculated  through  the 
test  set  up  for  the  duration  of  the  test.  The  sump  size  available  at  the  manufacturer’s 
facility  made  it  difficult  to  achieve  the  desired  conditions  to  obtain  the  flow  data,  since 
the  smaller  sump  size  resulted  in  turbulence,  aeration,  and  elevated  water  temperatures 
for  the  recirculated  water,  all  of  which  can  adversely  impact  the  results.  Because  of  these 
types  of  abnormalities  in  testing  pumps  of  this  size,  it  is  not  uncommon  within  the 
industry  to  gather  the  performance  data  from  a  scaled  model  rather  than  from  a  full-sized 
pump.  It  is  the  opinion  of  the  Parsons  team  that  results  from  the  factory  prototype  pump 
test  are  not  reliable  to  accurately  predict  flow  rates. 

2.4.3  Factory  Scaled  Model  Test 

At  the  request  of  the  USACE  HPO  in  New  Orleans,  ERDC  was  requested  to  observe  and 
assist  with  testing  a  scaled  model  (with  a  ratio  of  1:3.75)  of  the  MWI  pumps.  This  test 
was  performed  to  compare  the  measured  discharge  total  dynamic  head  of  the  scaled 
model  pump  with  the  design  values  for  the  prototype  pumps,  and  then  extrapolate  that 
data  to  predict  the  performance  of  the  prototype  pumps.  The  report  reviewed  for  this 
portion  of  the  evaluation  was  “Draft  Data  Report  on  Factory  Model  Tests  of  Discharge 
and  Head  of  MWI  Pumps  Used  on  New  Orleans  Outfall  Canals,”  September  2007. 

2.4.3. 1  Test  Setup 

The  following  are  the  details  given  in  this  report  that  were  checked  with  ANSI/HI 
standards  contained  in  ANSI/HI  2.6-2000: 

1.  Discharge  pressure  tap:  The  location  of  the  pump_discharge  pressure  tap  is 
consistent  with  the  dimensions  required  by  ANSI/HI  Figure  2.85,  page  29. 

2.  Flow  meter  location:  Distance  from  pump  discharge  head  fittings  is  greater  than  the 
minimum  required  by  ANSI/HI  Table  2.14,  page  26  for  a  throat/inlet  diameter  ratio 
of  0.718  therefore  acceptable. 

3.  Throttling  Valve  Location:  The  distance  from  the  Venturi  to  the  valve  is  more  than 
sufficient  per  ANSI/HI  Table  2.16,  page  27. 

4.  Return  to  Sump:  The  discharge  is  below  the  water  surface  to  prevent  aeration. 
Turbulence  generated  against  the  bottom  can  not  be  detennined. 

5.  Interior  Sump  Box:  The  report  states  that  the  interior  sump  box  was  based  on 
acceptable  sump  design  in  the  ANSI/HI  standards.  Assuming  a  model  pump  bell  of 
25.1  inches  (full-size  bell  diameter  of  7.85  ft  divided  by  the  model  scale  of  1:3.75), 
the  length  of  the  sump  box  was  shorter  and  the  width  was  slightly  narrower  than 
that  recommended  in  ANSI/HI  9.8-1998  Standard  for  Pump  Intake  Design.  The 
potential  impact  of  the  deviations  is  that  some  swirl  may  form  in  the  intake  that 
could  influence  the  head  or  flow  measurements,  but  such  influence  would  be  minor. 
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2.4.3. 2  Instrumentation 

1.  Mercury  manometer:  A  common  instrument  used  in  the  industry  for  accurate 
differential  pressure  readings  and  is  identified  in  this  report  for  measuring  static 
head. 

2.  Venturi  flow  meter:  This  is  a  typical,  accurate  flow  measuring  device  of  sufficient 
dimensions  to  give  accurate  readings.  There  is  no  indication  in  ERDC’s  report  as  to 
the  differential  pressure  device  used  to  take  readings.  The  report  indicated  Alden 
Research  Lab  had  performed  a  calibration  of  the  instrument,  but  the  calibration 
report  was  not  included  with  ERDC’s  report  nor  was  the  date  of  calibration  noted. 

3.  Tachometer:  A  laser,  hand-held  tachometer  was  used  and  readings  checked  against 
a  second  tachometer.  No  calibration  data  were  noted  for  either  instrument;  however, 
the  readings  from  each  instrument  checked  against  each  other. 

4.  Torque  Measurement:  This  is  also  a  common  means  of  measuring  input  power. 
Calibration  data  is  included  in  the  report  but  no  original  document  with  the  date  of 
calibration  was  included.  ANSI/HI  requires  torque  meters  be  calibrated  within 
certain  intervals.  It  is  the  opinion  of  the  Parsons  team  that  the  omission  of  this  data 
probably  will  have  minimal  affect  on  the  results. 

2.4.3.3  Findings 

The  test  setup  and  instrumentation  used  for  the  scale  model  were  consistent  with 
ANSI/HI  standard  tests,  with  the  exception  of  the  size  of  the  sump  box  as  noted  above. 
The  documentation  provided  for  the  test  did  not  include  all  the  detailed  data  described  in 
ANSI/HI  standards,  page  14,  “Summary  of  Necessary  Data  on  Pumps  to  be  Tested,”  but 
the  details  are  referenced  in  the  ERDC  report. 

A  check  of  the  data  recorded  in  the  report  showed  the  model  pump  efficiency  to  be 
calculated  correctly.  The  model  test  data  is  then  extrapolated  to  what  can  be  expected 
from  the  prototype  based  on  the  affinity  laws.  The  efficiency  calculations  assumed  an 
exponent  of  0.26  for  the  formula.  This  conservative  and  acceptable  number  is  related  to 
the  casting  roughness  of  the  model  compared  to  the  prototype.  The  report  did  not 
include  the  predicted  horsepower  requirements  for  the  prototype,  but  it  has  been 
calculated  and  added  to  Table  2-1  below. 

The  predicted  horsepower  is  important  to  determine  the  requirements  for  the  hydraulic 
drive  system  and  sizing  of  the  engine  power  supply  for  the  full-size  equipment.  Data  on 
the  efficiency  of  the  hydraulic  drive  system  was  provided  by  MWI.  The  Parsons  team 
used  the  efficiency  data,  as  published  by  the  component  manufacturers  and  compared  this 
data  to  the  required  HP.  The  results  of  this  evaluation  verify  the  adequacy  of  the 
components. 
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Table  2-1 — Predicted  Pump  Horsepower 


Test 

Prototype  Qa 
based  on 
affinity  laws, 

gpm 

Prototype 
TDH  at  288 
rpm  based 
on  affinity 
laws,  ft 

Prototype 

pump 

efficiency, 

Percent 

(%) 

Prototype 

pump 

horsepower, 

HP 

10 

64914 

22.54 

74.4 

497 

11 

81174 

18.41 

78.5 

483 

12 

85059 

16.95 

78.3 

467 

13 

96448 

12.1 

74.4 

399 

14 

100683 

9.26 

68 

348 

15 

104367 

7.59 

64.1 

313 

a  Q  is  flowrate 

b  TDH  is  Total  Dynamic  Head 

Ref:  Draft  Data  Report  on  Factory  Model  Tests  of  Discharge  and  Flead  of  MWI 
Pumps  Used  on  New  Orleans  Outfall  Pumps,  Table  3,  September  2007. 

2.4.4  Hydrostatic  Tests 

The  water  pump  units  were  tested  hydrostatically  for  90  minutes  to  check  for  leaks.  The 
process  included  raising  the  pressure  in  the  high-pressure  plumbing  (hose)  and  the  pump 
head  to  3200  psi  while  restraining  the  propellers  with  wood  blocking  to  induce  the  test 
pressure.  Hydrostatic  test  data  from  a  Jacksonville  QA  report  indicates  that  all  static 
tests  conducted  on  the  pump  units  successfully  met  the  specified  requirements. 

2.4.4.1  Findings 

All  of  the  pump  casings  passed  these  tests.  Records  show  some  pumps  tested  were  not 
initially  successful  and  that  these  pumps  went  through  corrections  and  further  testing. 
The  types  of  malfunctions  noted  in  the  reports  during  equipment  testing  are  considered 
nonnal  in  an  industrial  manufacturing  environment.  The  Parsons  team’s  opinion  is  that 
the  pumps  were  conclusively  tested  to  an  acceptable  operational  standard. 

2.5  Field  Testing 

The  field  testing  of  the  pumps  was  divided  into  performance,  acceptance,  and  endurance 
testing.  All  field  testing  occurred  at  the  three  pump  stations  in  New  Orleans. 

2.5.1  Performance  Testing 

Performance  testing  was  perfonned  in  the  field  to  measure  the  flow  capacity  of  the 
pumps  during  operation.  Parsons  conducted  a  review  of  each  field  test  by  comparing  the 
test  data  to  the  calculated  performance  of  the  system.  The  calculations  were  completed 
by  Parsons  using  the  installation  drawings  to  detennine  pipe  lengths,  number  of  elbows, 
and  elevations.  Parsons  did  not  conduct  a  detailed  evaluation  due  to  limited  actual 
installed  pump  system  head  and  flow  data  that  would  allow  verification  of  the  theoretical 
system  curves. 
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2. 5.1.1  17th  Street  Canal  Pump  Station 

The  report  reviewed  was  “Draft  Data  Report,  Field  Study  of  Pump  Discharge  and  Head  at 
17th  Street  Canal  Interim  Pumping  Station,”  May  2007. 

The  flow  determination  of  the  field  tests  is  dependent  on  the  operating  head  of  the  system. 
In  general,  the  system  curve  (head  versus  flow)  for  pumps  that  have  a  low  operating  head 
is  quite  flat.  Each  pump  was  fitted  with  piezometers  to  measure  the  head  produced  by 
each  pump.  No  reading  was  taken  on  the  manifold  so  it  did  not  facilitate  comparison  of 
system  pressure  with  the  system  flow  readings  from  the  acoustic  flow  meters  on  the 
manifold.  The  pump  curve  performance  noted  in  the  report  does  not  include  the 
operating  rpm  that  the  accuracy  could  not  be  checked  when  plotted  on  the  system  curve. 
It  is  the  opinion  of  the  Parsons  team  that  these  results  of  the  field  testing  for  the  17th 
Street  Canal  temporary  pump  station  does  not  provide  confirmation  of  the  capacity  of  the 
pumps,  but  does  provide  an  approximate  capacity  that  is  acceptable. 

2. 5.1.2  Orleans  Avenue  Canal  Pump  Station 

The  report  reviewed  was  “Draft  Data  Report,  Field  Study  of  Pump  Discharge  and  Head  at 
Orleans  Avenue  Canal  Interim  Pumping  Station,”  May  2007. 

The  performance  data  collected  for  Pumps  W5-1  and  W5-3  that  were  on  the  same  three 
pump  manifold  indicates  which  pump  discharge  head  was  approximately  2  feet  off  the 
theoretical  system  curve.  The  performance  data  collected  for  Pumps  W5-4  and  W5-5  that 
were  on  a  two  pump  manifold  identified  which  of  the  air  vacuum  breakers  did  not  seal 
and  allow  for  full  prime  of  the  siphons.  Therefore,  the  pumps  ran  at  a  higher  head  than 
would  be  expected.  On  this  basis,  this  test  provides  approximate  data.  It  is  the  opinion  of 
the  Parsons  team  that  these  results  of  the  field  testing  for  the  Orleans  Avenue  Canal 
Interim  Pump  Station  do  not  provide  confirmation  of  the  capacity  of  the  pumps,  but  does 
provide  approximate  capacities  of  the  pumps  that  is  acceptable. 

2. 5.1.3  London  Avenue  Canal  Pump  Station 

The  reference  report  was  “Draft  Data  Report,  Field  Study  of  Pump  Discharge  and  Head 
at  London  Avenue  Canal  Interim  Pumping  Station,”  July  2007. 

The  pump  curves  plotted  were  from  the  model  test  and  prototype  test  data  corrected  from 
188  rpm.  Data  plot  matches  the  corrected  model  test  data  curve,  but  the  head  was 
significantly  higher  than  the  theoretical  system  curve.  However,  data  from  three  different 
measuring  systems  did  not  correlate  with  any  of  the  curves.  Some  of  this  may  be  due  to 
the  lack  of  pressure  head  readings  on  the  manifold  where  the  flow  readings  were  taken. 
It  is  the  opinion  of  the  Parsons  team  that  these  results  of  the  field  testing  for  the  London 
Avenue  Canal  temporary  pump  station  do  not  provide  confirmation  of  the  capacity  of  the 
pumps,  but  does  provide  approximate  capacities  of  the  pumps  that  is  acceptable. 

2. 5.1.4  Findings 

The  field  perfonnance  tests  at  each  of  the  pump  stations  used  at  least  two  methods  of 
flow  measurement,  but  the  London  Avenue  station  appears  to  have  used  four  methods. 
One  of  the  methods  used  on  all  three  stations  was  piezometer  head  readings  transferred  to 
the  prototype  test  performance  curve  to  determine  flow.  This  data  method  is  approximate 
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since  there  was  no  way  to  accurately  detennine  impeller  rpm.  Thus,  the  field  tests  can  be 
considered  as  representing  the  approximate  capacity  these  stations  can  produce. 

Two  Rineer  motors  experienced  pulsating  activity  while  running  the  pumps  on  site. 
Investigations  revealed  there  was  a  problem  with  inadequate  stiffness  of  the  springs  in  the 
motors.  Once  those  springs  were  replaced,  the  pulsating  issue  was  resolved.  All  of  the 
Rineer  motors  onsite  today  have  the  new  springs  installed.  Inspection  of  these  motors 
should  be  part  of  the  normal  maintenance  activities  at  the  stations  so  if  any  wear  is  noted, 
it  can  be  addressed. 

2.5.2  Acceptance  Testing 

The  field  acceptance  tests  for  each  complete  system  included  running  at  least  2  hours  at 
an  engine  speed  of  1,800  rpm  and  a  hydraulic  pressure  of  3,200  psi.  Steady-state 
conditions,  engine  rpm,  engine  jacket  temperature,  hydraulic  system  oil  pressure  and 
temperature,  leakage  (required:  none),  and  canal  level  were  monitored.  These  tests  were 
conducted  on  each  hydraulic  pumping  system  by  the  contractor  with  oversight  by 
USACE.  USACE  documented  any  deviations  from  the  testing  parameters  including 
pump  speeds,  run  times  and  temperatures. 

The  documentation  showed  all  abnormalities  previously  identified  in  the  pump 
manufacture  and  installations  were  corrected  prior  to  the  acceptance  tests.  All  HPUs 
were  fitted  with  new  cams  on  site  in  July  2006  prior  to  acceptance  tests.  Reports  by 
USACE  showed  that  acceptance  inspections  started  in  June  2007,  and  included  punch 
lists  for  the  drive  units  noting  the  physical  abnonnalities  to  be  corrected.  Most 
abnonnalities  were  corrected  by  September,  2007,  with  a  few  minor  issues  still  noted  in 
the  punch  list  for  the  drive  units.  A  review  of  the  acceptance  test  results  of  the  pumps 
follows. 

2. 5.2.1  London  Avenue 

Acceptance  tests  for  the  London  Avenue  Canal  pumps  started  in  July  2007.  The  test 
results  indicate  that  fully  loaded  run  tests  were  perfonned  on  12  pumps  at  the  site.  Out  of 
the  12  pumps  tested,  9  pumps  passed  the  initial  acceptance  tests.  Functional 
abnonnalities  such  as  oil  leaks,  high  oil  temperature,  and  overheating  gear  oil  caused  the 
3  pumps  to  fail  the  initial  tests.  These  abnormalities  were  corrected  and  the  3  pumps  then 
passed  the  running  test  as  shown  on  the  pump  acceptance  log,  dated  November  2007. 

2. 5.2. 2  Orleans  Avenue 

Acceptance  tests  for  the  Orleans  Avenue  Canal  pumps  started  in  June,  2007.  Functional 
abnormalities,  including  a  damaged  seal,  leaking  bearing  o-ring  and  underwater  oil  leaks, 
occurred  with  3  pumps.  These  abnormalities  were  corrected  by  August,  2007  and  the 
pumps  were  re -tested.  All  passed  the  running  test  as  shown  on  the  pump  acceptance  log, 
dated  November,  2007. 

2. 5.2.3  17th  Street 

Acceptance  tests  for  the  17th  Street  Canal  pumps  started  in  August,  2007.  The  test  logs  of 
September,  2007  indicate  10  out  of  18  pumps  underwent  the  fully  loaded  test  and  all  10 
pumps  passed.  No  functional  abnonnalities  occurred  with  these  ten  pumps.  The 
remaining  8  pumps  were  tested  by  September,  2007  with  all  8  pumps  passing.  Quality 
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Assurance  Reports  show  that  due  to  low  canal  levels,  some  pumps  were  run  at  reduced 
speeds  of  1400  rpm  and  some  pumps  were  tested  for  shorter  periods  of  1.5  hours  during 
pump  tests.  These  pumps  were,  however,  also  deemed  to  have  passed  the  acceptance  tests 
by  the  USAGE  Quality  Assurance  Team  because  perfonnance  was  demonstrated  upon 
reaching  45  minutes  of  steady  state  conditions. 

2. 5.2.4  Findings 

All  40  pump  systems  were  finally  accepted.  It  is  the  opinion  of  the  Parsons  team  that 
there  was  due  diligence  in  the  inspection  and  correction  of  any  functional  abnonnalities 
throughout  the  testing.  Abnormalities  encountered  were  normal  to  the  commissioning 
and  startup  of  this  type  of  equipment. 

2.5.3  Endurance  Testing 

An  endurance  test  was  performed  on  a  drive  unit  at  the  London  Canal.  This  test  used  a 
HPU  mounted  to  the  bridge  upstream  of  the  flood  gate  with  a  spare  pump  mounted  to  the 
temporary  sheet  pile  structure.  This  temporary  sheet  pile  structure  was  installed  as  a 
contingency  plan  pending  completion  of  the  temporary  pump  station.  During  the  initial 
attempts  to  run  the  test,  functional  abnormalities  with  the  drive  unit  had  to  be  corrected. 
Once  those  corrections  were  made,  the  HPU  successfully  completed  a  total  of  37  hours, 
29  minutes  of  operation. 

2.5.3.1  Findings 

The  evaluation  of  the  endurance  test  report  showed  that  there  were  three  attempts  to  run 
the  endurance  test.  Typical  startup  abnormalities  were  encountered  with  the  first  two 
attempts.  On  the  third  attempt  the  pump  successfully  performed  without  incident. 

2.6  Laboratory  Physical  Sump  Model  Testing 

A  pump  of  this  size  and  type  can  be  sensitive  to  the  approach  flow  hydraulics,  and 
keeping  with  ANSI/HI  standards,  the  HPO  engaged  the  USAGE  ERDC  in  Vicksburg,  MS, 
to  conduct  a  physical  model  study  of  the  Interim  Pumping  Stations  at  both  the  London 
Avenue  Canal  and  17th  Street  Canal.  No  physical  model  study  was  conducted  of  the 
Orleans  Avenue  Canal’s  pump  station  sump,  as  ERDC  determined  the  modifications 
developed  for  the  London  Avenue  Canal  would  be  effective  at  the  Orleans  Avenue 
Canal’s  temporary  pump  station  because  the  approach  canal  layouts  were  similar.  The 
intake  modifications  which  were  developed  for  the  London  Avenue  station  were 
replicated  at  the  Orleans  Avenue  station. 

Two  reports  were  developed  for  the  physical  model  studies.  The  first  report,  “Physical 
Model  Study  of  Interim  Pumping  Station  at  London  Avenue  Canal,  New  Orleans, 
Louisiana,”  dated  January  2008  covers  the  study  perfonned  for  the  London  Avenue 
Canal  station.  The  second  model  study  covered  the  17th  Street  Canal  station  and  is  titled 
“Physical  Model  Study  of  Interim  Pumping  Station  at  17th  Street  Canal,  New  Orleans, 
Louisiana,”  January  2008.  The  model  studies  were  used  to  evaluate  the  potential  for 
surface  and  subsurface  vortices,  flow  pre-swirl  entering  the  pumps,  and  the  velocity 
distribution  at  the  pump  impeller  location.  The  Parsons  team  reviewed  the  model  studies 
with  ERDC  in  New  Orleans  in  November  2008. 
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The  review  of  the  report  and  meeting  are  summarized  in  the  following  subsections. 

Typically,  the  sump  model  studies  are  conducted  as  part  of  the  design  of  the  station  and 
HI  states  that  for  stations  of  this  size  and  criticality  they  are  required.  Due  to  the  very 
rapid  timeline  associated  with  this  installation,  the  conventional  timelines  associated  with 
physical  models  of  pumps  were  not  able  to  be  accommodated;  therefore,  ERDC 
conducted  the  model  study  after  the  pumps  were  installed  and  operating  at  the  site.  The 
modifications  that  were  developed  as  a  result  of  the  model  testing  were  then  subsequently 
installed  at  the  London  Avenue  and  Orleans  Avenue  pump  stations. 

2.6.1  Model  Scale 

The  selection  of  model  scale  is  based  on  minimizing  the  fluid  viscous  effects  (Reynolds 
number)  and  surface  tension  effects  (Weber  number).  The  model  scale  that  was  selected 
for  these  pumps,  1:15,  resulted  in  a  pump  bell  Reynolds  number  of  6  x  104,  which  meets 
the  HI  criterion.  This  scale  also  satisfied  internal  ERDC  criterion.  No  mention  was  made 
in  the  report  of  whether  a  1:15  scale  model  met  the  Weber  number  criterion;  however, 
Parsons  team  calculations  determined  that  it  was  acceptable. 

2.6.2  Model  Layout  and  Extent 

The  models  for  the  two  stations  included  all  pumps  (hydraulic  and  direct  drive),  pump 
platforms,  and  support  structures  that  were  in  the  water.  The  model  pump  bells  simulated 
the  prototype  pump  bells.  Approach  channels,  1,100  feet  for  London  Avenue,  840  feet 
for  17th  Street,  were  simulated  in  the  models.  The  canal  widths  of  360  feet  and  240  feet 
respectively  for  the  London  Avenue  and  17th  Street  Canals  make  the  approach  channels 
somewhat  short,  but  since  the  canals  are  relatively  straight  upstream  of  the  pump  stations 
and  an  overflow  weir  was  used  at  the  upstream  end  of  the  models,  the  model  length  is 
acceptable.  The  only  issue  that  was  noted  was  the  flow  over  the  model  weir  caused 
surface  turbulence  that  in  fact  could  inhibit  the  formation  of  surface  vortices  which  may 
occur  in  the  prototype,  resulting  in  less  conservative  results. 

Llow  calibration  of  the  individual  model  pumps  followed  generally  accepted  practice. 

Canal  bathymetry  (a  study  of  underwater  depths),  to  measure  depths  of  the  canal,  was 
developed  using  gravel.  This  was  used  to  simulate  the  canal  roughness  since  the  canal  in 
the  area  of  the  pump  station  was  protected  with  rock;  therefore  using  gravel  to  simulate 
roughness  was  a  reasonable  approach. 

2.6.3  Test  Conditions 

Since  there  are  no  test  standard  requirements  to  assist  with  detennining  the  critical  pump 
operating  combinations,  professional  judgment  was  used.  ERDC  decided  to  conduct 
three  test  conditions,  which  were  as  follows:  (1)  all  pumps  operating  (hydraulically 
driven  and  direct  drive);  (2)  all  hydraulic  drive  pumps  operating;  and  (3)  all  direct  drive 
pumps  operating.  In  addition,  for  the  17th  Street  station,  a  fourth  condition  was  tested, 
which  consisted  of  all  the  MWI  hydraulically  driven  pumps  and  the  portable  pumps  on 
the  interim  gated  structure,  which  were  providing  an  additional  1400  cfs  capacity.  More 
typical  operations  when  fewer  pumps  are  operating  were  not  simulated  with  the  models, 
but  it  was  the  opinion  of  ERDC  that  the  most  extreme  operating  condition  is  when  the 
stations  are  running  at  full  capacity.  This  may  be  a  non-conservative  opinion  since  the 


Final  Report 

2-12 


February  27,  2009 


Independent  Engineering  Assessment  of  the 

New  Orleans  Temporary  Outfall  Canal  Pumps  Testing  Adequacy  Analysis 


worst  approach  flow  hydraulics  can  occur  when  fewer  pumps  are  operating.  For  example, 
the  water  surface  was  turbulent  during  the  model  tests  which  could  have  inhibited  the 
formation  of  surface  vortices.  With  less  pumps  operating,  the  water  surface  in  the  model 
will  be  more  tranquil  which  is  conducive  to  the  formation  of  surface  vortices.  With  more 
pumps  operating,  the  approach  velocities  are  higher  which  may  wash  out  subsurface 
vortices.  Again,  with  less  pumps  running,  the  approach  velocities  will  be  less  and  may 
allow  for  subsurface  vortices  to  become  stronger  and  more  stable. 

2.6.4  Instrumentation 

The  instrumentation  used  in  the  model  studies  followed  nonnal  laboratory  procedures 
and  ANSI/HI  standards.  The  velocity  fluctuation  data  at  the  pump  impeller  location  was 
not  taken  (discussed  below).  Swirl  meters  were  installed  according  to  ANSI/HI  standards 
and  flow  velocities  were  taken  at  several  locations  which  were  selected  based  on  high 
flow  pre-swirl  readings.  This  is  normal  practice. 

2.6.5  Acceptance  Criteria 

The  acceptance  criterion  was  discussed  at  length,  with  ERDC  accepting  a  relaxed  HI 
standard  due  to  “infrequent  pump  operating  conditions”  (see  ANSI/HI  standards  criteria 
for  “infrequent  pump  operating  conditions”  as  stated  in  the  ERDC  January  2008  Report). 
The  infrequent  operation  conditions  that  the  HI  discusses  are  usually  associated  with  a 
station  operating  under  abnonnal  conditions.  During  normal  operation  the  station  should 
meet  the  stricter  criteria.  The  interpretation  used  by  ERDC  is  that  since  these  stations 
only  operate  infrequently,  the  use  of  the  relaxed  standards  for  “infrequent  pump  operating 
conditions”  was  appropriate.  In  addition,  ERDC  only  conducted  tests  at  extreme  flow 
conditions  and  more  frequent  operating  conditions  were  not  analyzed.  It  is  unknown 
whether  the  pump  stations  would  meet  the  stricter  criteria  for  more  nonnal  operations. 
Since  a  scale  model  is  nonconservative  related  to  the  formation  of  surface  or  subsurface 
vortices  due  to  viscous  effects,  applying  the  more  relaxed  HI  standard  associated  with 
infrequent  operation  could  result  in  vortices  entering  the  pumps  that  have  a  low  enough 
pressure  to  result  in  vibration,  cavitation,  and  bearing  wear. 

One  HI  criterion  related  to  measuring  time -varying  velocities  at  the  pump  impeller 
location  was  not  evaluated  due  to  time  limitations  and  the  fact  that  ERDC  rarely  takes 
these  measurements.  The  time -varying  velocity  measurement  can  pick  up  excessive 
velocity  fluctuations  that  could  result  in  flashes  of  cavitation  and  bearing  wear.  However, 
research  has  indicated  that  exceeding  the  criterion  does  not  normally  result  in  significant 
pump  performance  issues. 

HI  also  states  that  a  few  tests  with  the  final  design  should  be  conducted  at  1.5  Froude 
scaled  flow  rates.  The  Froude  number  is  defined  as  the  characteristic  velocity  divided  by 
the  water  wave  propagation  velocity.  This  will  compensate  for  any  possible  scale  effects 
on  free-surface  vortices.  However,  when  it  is  determined  the  flow  patterns  are  too 
distorted  to  provide  a  reasonable  analysis,  it  is  not  necessary  to  perform  these  tests.  In 
this  case  these  tests  were  not  performed  for  this  reason. 

The  ERDC  report  states  that  one  rationale  used  for  accepting  the  relaxed  standards  is  based 
on  satisfactory  prototype  test  results  of  a  one  day  test  which  was  conducted  in  the  field  in 
April,  2007.  During  these  field  tests,  the  formation  of  surface  vortices  were  observed.  A 
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scale  model  test  was  conducted  with  the  same  operation  scenario  and  indicated  that  both 
surface  and  subsurface  vortices  formed  and  that  flow  pre-swirl  occurred  in  excess  of 
acceptable  levels. 

It  should  be  noted  that  experience  indicates  that  almost  all  the  hydraulic  issues  associated 
with  poor  approach  flow  hydraulics  do  not  cause  instantaneous  abnormalities.  Typically, 
the  effect  of  surface  and  subsurface  vortices,  along  with  pre-swirl  conditions,  results  in 
vibration  damage,  cavitation  damage,  and  bearing  wear  over  a  period  of  time  of  use. 

2.6.6  Test  Results 

Based  on  the  modified  HI  perfonnance  criteria,  it  is  Parsons  team’s  opinion  that  the 
model  studies  were  conducted  in  accordance  with  accepted  engineering  standards. 
Modifications  to  the  sump  designs  were  developed  using  the  models.  For  the  London 
Avenue  station,  the  modifications  to  reduce  swirl  consisted  of  the  installation  of  a  sump 
floor  cone  with  4  vanes  that  were  attached  to  the  pump  bells  and  hung  below  the  bells  to 
within  a  maximum  of  6  inches  above  the  sump  floor.  In  addition,  a  surface  vortex  grate 
was  installed  horizontally  just  below  minimum  water  level  to  address  submerged  vortex 
concerns.  These  same  modifications  were  installed  at  the  Orleans  Avenue  pump  station. 

For  the  17th  Street  pump  configuration,  the  modifications  identified  by  the  model 
included  lengthening  the  pump  bays  (for  west  and  east  side  6  pump  platforms)  and 
installing  grating  at  the  entrance  to  turn  the  flow  into  the  pump  bays.  No  modifications 
were  developed  to  control  submerged  vortices  since  they  were  considered  weak. 
However,  experience  shows  the  vortices  should  be  monitored  and  the  pumps  inspected 
according  to  a  regular  maintenance  program  to  ensure  that  there  are  no  signs  of  impeller 
or  bearing  deterioration. 

2.6.7  Implementation  of  the  Modifications  to  the  Prototype  Structures 

It  is  the  Parsons  team’s  understanding  that  the  modifications  identified  during  the  model 
testing  have  been  installed  at  the  London  Avenue  station  and  duplicated  at  the  Orleans 
station.  For  the  17th  Street  pump  station,  the  modifications  that  were  developed  on  the 
model  were  not  implemented  in  the  field  due  to  the  extent  of  the  modifications  identified. 
There  is  ample  redundancy  at  this  station  and  other  pumps  can  be  brought  on  line  if  there 
are  abnormalities  with  some  of  the  pumps.  The  redundancy  also  allows  for  the  operating 
sequence  that  rotates  the  cycle  time  on  the  pumps. 

2.6.8  Findings 

The  sump  model  studies  for  the  London  Avenue  and  17th  Street  Canals’  interim  pump 
stations  generally  met  ANSI/HI  standards.  The  performance  criteria  that  were  not 
evaluated  or  were  relaxed  may  have  an  impact  on  the  performance  of  the  pumps; 
however,  it  is  the  opinion  of  the  Parsons  team  that  the  modifications  that  were  developed 
in  the  model  should  minimize  any  effect,  especially  if  a  thorough  inspection  and 
maintenance  program  is  followed. 

Not  constructing  and  installing  the  recommended  modifications  at  the  17th  Street  station 
may  result  in  performance  issues,  such  as  cavitation,  vibration,  and  bearing  wear.  It  is 
recommended  that  the  pumps  be  inspected  for  unusual  wear  as  part  of  a  regular 
maintenance  program.  If  there  are  signs  of  deterioration,  then  the  recommended 
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modifications  from  the  model  study  could  be  implemented  to  improve  pump  performance 
and  longevity. 

Although  the  Orleans  Avenue  station,  which  includes  the  modifications  developed  for  the 
London  Avenue  station,  should  perform  satisfactorily,  this  station  has  no  backup  system 
(i.e.,  direct  drive  pumps  nor  excessive  flow  capacity).  To  confirm  the  design,  a  model 
study  could  be  conducted  to  verify  the  effectiveness  of  the  modification.  As  an  option, 
the  pumps  at  this  station  could  be  monitored  for  vibration  and  pulled  and  inspected  for 
wear  as  part  of  a  regular  maintenance  program. 

2.7  Conclusions 

It  is  the  Parsons  team’s  opinion  that  the  scale  pump  testing  at  the  factory,  and  the  sump 
model  testing  for  the  17th  Street  and  London  Avenue  Canal  Interim  Pump  Stations,  are 
adequate  for  their  intended  purpose.  The  scale  pump  test  should  be  considered  the 
definitive  test.  No  additional  pump  testing  is  required.  For  the  Orleans  Avenue  Canal 
Interim  Pump  Station,  a  pump  sump  physical  model  study  was  not  conducted  as  specified 
by  HI. 

2.7.1  Factory 

The  testing  performed  on  the  pump  systems  is  in  line  with  industry  standards. 
Abnormalities  identified  during  the  testing,  both  at  the  factory  and  in  the  field,  were 
satisfactorily  addressed.  Additional  testing  was  performed  in  the  field  to  verily  the  pump 
performance. 

Due  to  the  accelerated  procurement  schedule,  acceptance  testing  at  the  factory  was 
started  prior  to  the  manufacturer  being  able  to  troubleshoot  the  equipment  set  up, 
resulting  in  the  onsite  inspectors  observing  an  increased  number  of  incidents  with  the 
equipment.  Usually  the  manufacturer  has  had  time  to  set  up  a  system  and  perform  a 
mock  test  to  ensure  that  the  equipment  will  perform  as  anticipated  when  the  inspectors 
arrive.  The  testing  of  each  drive  unit  at  the  factory  proved  very  valuable  in  that 
abnormalities  in  the  cams  and  other  mechanical  components  were  able  to  be  identified 
and  corrected  before  the  field  testing  occurred,  greatly  reducing  the  amount  of  time 
normally  associated  with  commissioning  and  acceptance  testing. 

Full-size  testing  of  pumps  of  this  size  is  not  usually  required,  as  a  model  is  normally  used 
to  obtain  performance  parameters  for  such  large  pumps.  The  testing  of  each  individual 
pump  in  the  factory  is  above  what  is  normally  specified,  too,  since  all  the  pumps  are 
manufactured  to  the  same  specifications  and  will,  therefore,  have  little  change  in 
performance  from  pump  to  pump. 

With  regard  to  pump  capacity,  it  is  the  Parsons  team’s  opinion  that  the  use  of  the  scaled 
model  test,  extrapolated  to  predict  the  perfonnance  of  the  prototype  pump  is  a  better 
representation  of  the  pump  capacity  than  the  prototype  pump  testing  results  that  were 
witnessed  by  ERDC. 

Despite  the  anomalies  witnessed  during  these  factory  tests,  the  Parsons  team’s  opinion  is 
that  the  factory  testing  was  adequate  for  its  intended  purpose  and  the  response  to 
correcting  the  failures  indicates  diligence  was  exercised.  Additionally,  the  field  testing  to 
verify  satisfactory  performance  supersedes  the  events  witnessed  at  the  factory. 
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2.7.2  Field 

The  acceptance  and  endurance  testing  in  the  field  was  performed  in  general  confonnance 
with  industry  standards.  The  anomalies  experienced  during  the  acceptance  testing  are 
consistent  with  the  types  of  anomalies  normally  experienced  during  the  startup  phase  of 
pennanent  pump  stations  designed  and  constructed  for  USAGE.  Correction  and  retesting 
is  typically  administered  until  the  witnessed  anomalies  are  eliminated  and  there  are  no 
other  anomalies  experienced.  The  acceptance  testing  documentation  indicates 
consistency  with  this  industry  standard. 

2.7.3  Laboratory 

The  laboratory  testing  was  in  general  confonnance  with  the  applicable  ANSI/HI 
standards.  The  report  discusses  the  acceptance  criteria  used  for  the  evaluation  of  the  test 
results.  The  temporary  pump  stations  at  the  subject  canals  are  unique  in  that  their 
required  use  is  only  during  those  design  rainfall  events  during  which  Lake  Pontchartrain 
experiences  stonn  surge.  This  interpretation  versus  the  more  strict  interpretation  of 
“infrequent  pump  operating  conditions”  has  a  material  effect  on  the  detennination  of 
whether  the  observed  vortices  during  the  model  tests  are  acceptable  or  not.  Regardless 
of  the  interpretation,  the  consequences  of  the  approach  hydraulic  conditions  observed  are 
typically  time  dependent  for  any  substantive  mechanical  degradation.  Moreover,  these 
tests  are  typically  conducted  in  advance  of  construction  to  provide  an  opportunity  to 
modify  pump  conditions  before  the  construction  activities.  The  emergency  response  for 
construction  did  not  lend  itself  to  this  typical  sequencing. 


Final  Report 

2-16 


February  27,  2009 


Independent  Engineering  Assessment  of  the 
New  Orleans  Temporary  Outfall  Canal  Pumps 


Vulnerability  Analysis 


Section  3  -  Vulnerability  Analysis 
3.1  Purpose 

An  assumption  in  a  vulnerability  analysis  is  that  mechanical  systems  are  inherently 
vulnerable  to  breakdowns.  The  purpose  of  this  effort  is  not  to  predict  the  absence  of 
future  breakdowns,  nor  to  predict  timing  of  future  breakdowns,  but  rather  to  identify  any 
conditions  that  would  indicate  extraordinary  and  unacceptable  vulnerabilities  leading  to 
flooding  of  areas  to  be  protected  by  the  temporary  pumping  facilities.  Equipment  failures 
are  inherent.  Excessive  equipment  failures  and  lack  of  ability  to  accommodate  nonnal 
failures  are  unacceptable  vulnerabilities.  The  Parsons  team’s  investigation  indicates  that 
planning,  design,  equipment  procurement,  and  construction  for  project  delivery  were 
conducted  under  extreme  conditions.  The  planning,  design,  procurement,  construction, 
installation,  and  acceptance  of  over  16,600  cfs  in  pumping  capability  in  a  period  of 
approximately  2 1  months  required  substantial  deviation  from  nonnal  procurement,  design, 
and  construction  protocols.  Substantially  witnessed  abnormalities  of  the  pumping 
systems  were  observed  during  the  fabrication  and  assembly  processes.  As  discussed  in 
previous  sections  these  types  of  tests  are  not  typically  witnessed  by  the  purchaser  as  they 
are  identified  and  resolved  by  the  fabricator  before  the  inspection  effort.  Therefore,  the 
observations  by  the  witnesses  during  fabrication  are  considered  in-progress  observations. 
Witnessed  underperfonnance  and  the  resolution  of  those  conditions  were  considered  in 
this  report.  The  Parsons  team’s  vulnerability  analysis  focuses  on  the  final  configuration 
and  performance  records  of  the  system  as  it  currently  exists  at  all  three  canal  outfall 
locations,  which  is  a  better  indication  of  actual  vulnerability  of  the  system. 

The  vulnerability  analysis  was  conducted  to  identify  extraordinary  conditions  of  the 
hydraulic  pumping  systems  and  their  supporting  systems  that  might  indicate  a 
vulnerability  of  failure  above  and  beyond  those  that  are  inherently  associated  with 
electro-mechanical  systems  as  applied  under  conditions  of  similar  risks  and  consequences 
of  failure.  Special  consideration  was  given  to  conditions  that  might  lead  to  under¬ 
performance  of  the  system  during  emergency  response  events.  The  scope  and 
assumptions  for  the  vulnerability  analysis  is  as  follows: 

•  The  temporary  pump  stations  are  designed  for  a  5-  to  7-year  service  life  because 
they  will  be  replaced  with  pennanent  pump  stations  by  2013. 

•  It  is  assumed  that  USACE  will  maintain  the  canals  at  their  designated  safe  level  in 
coordination  with  the  SWB. 

•  The  pump  stations  are  manned  during  a  stonn  and  hurricane  event. 

•  This  vulnerability  analysis  does  not  include  analysis  of  risks  related  to  canal 
levels,  flooding  risks,  flood  walls  integrity,  and  any  other  risk  factors  that  do  not 
relate  to  the  outfall  canal  temporary  hydraulic  pumping  systems. 

•  Design  and  construction  information  was  obtained  from  specifications,  design  and 
construction  drawings,  and  information  received  during  interviews  as  outlined 
below  in  Section  3.2.  Methodology. 
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3.2  Methodology 

This  section  outlines  the  methodology  used  to  analyze  potential  vulnerabilities.  The 
Parsons  team  conducted  this  assessment  by  evaluating  the  available  documents  and 
conducting  interviews  with  the  appropriate  agencies  and  individuals  with  regard  to  the 
pump  material/component  assessment,  structural  considerations,  electrical  and 
instrumentation  systems,  and  hydraulic  considerations.  After  the  interviews  and  site 
visits,  the  Parsons’  team  met  to  develop  a  list  of  “what  ifs.”  The  objective  of  this 
brainstonning  meeting  was  to  develop  a  list  of  threats  to  the  hydraulic  pumping  systems 
and  their  supporting  systems  with  the  goal  of  determining  potential  vulnerabilities  of  the 
system  under  analysis.  Through  this  process,  the  Parsons  team  prepared  a  database  of 
information  that  was  analyzed  to  determine  the  degree  of  vulnerability  utilizing  common 
engineering  practice.  This  process  takes  into  account  both  the  category  of  vulnerability 
and  the  category  of  criticality  of  the  asset  or  condition  being  analyzed.  (Refer  to  Table  3- 
1,  shown  below.)  The  distinction  of  these  two  elements  are  summarized  and  illustrated  as 
follows: 

•  Vulnerability  Category:  The  degree  to  which  the  element  being  analyzed  is 
vulnerable.  This  category  has  three  degrees  of  measure — extreme,  moderate,  and 
slight,  which  are  labeled  a,  b,  and  c  respectively. 

•  Criticality  Category:  The  degree  to  which  the  element  is  considered  critical  to 
operational  readiness.  This  category  has  three  degrees  of  measure — high, 
medium,  and  low,  which  are  labeled  1,  2,  and  3  respectively. 


Table  3-1 — Criticality 


Categories  of 
Vulnerability 

Categories  of  Criticality 

1  -  High 

2  -  Medium 

3  -  Low 

a  -  Extreme 

la 

2a 

3a 

b  -  Moderate 

lb 

2b 

3b 

c  -  Slight 

lc 

2c 

3c 

The  Asset  Risk  Level  is  therefore  determined  as  shown  below  in  Table  3-2: 


Table  3-2 — Asset  Risk  Levels 


Asset  Risk  Levels 

la 

2a 

lb 

High-Level  Priority  items  to  address 

lc 

2b 

2c 

Mid-Level  Priority  but  Agency  may  accept 

3a 

3b 

3c 

Low-Level  Priority  acceptable  to  Agency 

The  following  actions  were  conducted  to  evaluate  the  vulnerabilities  of  the  outfall 
temporary  hydraulic  pumping  systems  and  their  support  systems. 

•  Conducted  assessments  from  early  November  2008  through  December  2008.  The 
Parsons  team  reviewed  specifications  written  for  the  procurement  of  the  pumps, 
the  construction  documentation,  factory  tests  records,  and  performance 
requirements. 
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•  Interviewed  the  USAGE  project  officials  on  November  4,  2008,  at  the  New 
Orleans  District,  and  the  canal  operation  staff. 

•  Visited  the  sites  from  November  4  to  5,  2008,  of  the  three  temporary  pump 
stations  at  17th  Street,  London  Avenue,  and  Orleans  Avenue,  and  examined 
components  of  the  pumps,  pump  drives,  electrical  distribution,  and  control 
including  the  hydraulic  driven  gates. 

•  Observed  the  hydraulic  pumps  running  at  the  17th  Street  pump  station  on 
November  4,  2008,  and  interviewed  operational  staff. 

•  Visited  the  New  Orleans  SWB  Pump  Station  Number  6,  on  November  5,  2008,  to 
observe  operations  and  to  interview  operational  staff.  Interviewed  the  SWB 
Superintendent  and  staff. 

•  Interviewed  ERDC  on  November  6,  2008,  on  the  modeling  of  the  pump  station 
and  attended  a  presentation  by  ERDC  followed  by  a  question  and  answer 
discussion  on  the  science  of  his  methods  of  modeling. 

•  Visited  the  factory  facilities  of  the  pump  manufacturers  MWI  on  November  18, 
2008.  Met  with  production  management  and  the  chief  mechanical  engineer  and 
conducted  interviews  with  each  individual. 

•  Interviewed  the  USACE  Jacksonville  District  inspecting  officials  on  November 
20,  2008,  regarding  the  pump  testing  procedures.  We  discussed  their  observations, 
their  records,  and  their  opinions  of  the  testing  of  the  pumps  and  pump  drives. 

•  Held  brainstorming  sessions  between  November  8  and  28,  2008,  with  Parsons 
professional  engineering  team  regarding  the  vulnerabilities  to  failure  of  the 
temporary  hydraulic  pump  systems  and  their  supporting  systems. 

•  Conducted  follow  up  interviews  with  the  USACE  staff  in  New  Orleans  on 
December  2  and  3,  2008.  This  session  focused  on  the  USACE  equipment 
preventive  maintenance  program  in  place  and  additional  discussions  on  the 
acceptance  testing  timeline. 

•  Interviewed  the  USACE  whistleblower  on  January  28,  2009,  regarding  the  pump 
testing  procedures.  Discussions  were  held  regarding  the  whistleblower’s 
observations,  records  and  opinions  of  the  pump  and  pump  drive  testing. 

3.3  Findings  in  Brief 

The  vulnerability  analysis  was  conducted  considering  structural,  mechanical,  electrical, 
maintenance,  and  operational  aspects  of  the  hydraulic  pumping  system  including  its 
support  systems.  As  stated  in  subsection  3.1,  Purpose,  this  effort  is  intended  to  identify 
what,  if  any,  extraordinary  vulnerabilities  exist  for  the  hydraulic  pumping  systems  and  its 
supporting  systems.  In  summary,  it  is  the  Parsons  team’s  opinion  that  the  hydraulic 
pumping  systems  and  their  support  systems  do  not  exhibit  unusual  or  extraordinary 
vulnerabilities.  Table  3-3  indicates  the  vulnerabilities  identified  and  their  assessed  risk 
level  as  determined  through  the  previously  described  methodology  and  Table  3-1  and 
Table  3-2. 
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3.4  Component  Vulnerabilities 
3.4.1  Findings 

This  section  identifies  any  components  of  the  temporary  pump  stations  and  their 
support  systems  that  are  vulnerable  to  failure  before,  during,  and  after  a  storm  event. 


Table  3-3 — Component  Vulnerabilities 


Source  or  Cause  of 
Potential  Vulnerability 

Affected  Pump  Stations 

Assessed  Categories 

17th  Street 

Orleans 

London 

Criticality 

Vulnerability 

Protection  of  exposed  hoses 
engines  of  hydraulics, 
electrics  and  batteries  from 
wind  borne  debris, 
projectiles 

X 

X 

X 

3 

b 

Power  poles  next  to  diesel 
tanks 

X 

3 

b 

Elbow  fittings  Aeroquip  - 
leak  issue 

X 

X 

X 

2 

c 

Unsuitable  CCTV  camera 
mountings 

X 

X 

X 

3 

c 

Vapor  proof  lighting  in  the 
pump  area  and  under  the 
pump  deck 

X 

X 

X 

3 

c 

Unpainted  hydraulic  oil 
piping 

X 

X 

X 

2 

c 

Unsuitable  pole  mounted 
light  fixtures 

X 

X 

X 

3 

c 

Criticality:  1  -  High;  2  -  Medium;  and  3  -  Low 

Vulnerability:  a  -  Extreme;  b  -  Moderate;  and  c  -  Slight 

Note:  Refer  to  Tables  3-1  and  3-2  for  more  information. 

•  Protection  of  exposed  hoses  of  hydraulics.  Electrics  and  batteries  from  wind 
borne  debris  and  projectiles:  The  team  identified  that  these  components  are 
vulnerable,  but  they  are  not  detrimental  to  the  overall  operation  of  the  pump 
stations.  This  vulnerability  is  rated  low  on  the  scale  of  priorities  for  improvement. 

•  Power  poles  next  to  diesel  tanks:  A  power  pole  was  found  erected  next  to  one 
of  the  main  fuel  storage  tanks.  The  pole  carries  a  4,160  volt  step-down 
transformer  to  feed  the  17th  Street  Canal  temporary  outflow  pump  station.  The 
fuel  tank  is  in  the  path  of  the  pole  falling  radius.  The  team  rated  this  issue  as  a  low 
criticality,  but  it  has  an  average  rating  of  vulnerability. 

•  Aeroquip  elbow  fittings  leaks:  This  item  is  inserted  into  the  vulnerability  matrix 
with  the  intention  that  this  issue  be  kept  in  view  until  resolved  by  the 
manufacturer. 

•  Unsuitable  closed  circuit  television  (CCTV)  camera  mountings:  The  camera 
mountings  are  attached  to  the  poles  using  large  hose  clips.  These  hose  clips  are 
not  hurricane  rated.  U-bolts  are  typically  used  to  secure  the  cameras  subject  to 
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high  wind  conditions.  The  concern  here  is  that  the  cameras  could  come  loose  in  a 
hurricane  event  and  damage  critical  parts  of  the  hydraulic  pumping  system. 

•  Vapor  proof  lighting  fixtures  in  the  pump  area  and  under  the  deck  are  not 
impact  resistant:  The  electrical  lighting  was  examined,  and  it  was  noted  that  the 
lighting  fixtures  underneath  the  engine  platforms  were  of  the  vapor  proof 
fluorescent  type  fixtures.  These  fixtures  are  for  wet  areas,  but  they  are  not 
designed  for  hurricane  force  conditions.  Therefore,  they  could  potentially  fall  and 
destroy  critical  components  of  the  pump. 

•  Unpainted  hydraulic  oil  piping:  This  item  is  inserted  into  the  vulnerability 
matrix  with  the  intention  that  this  issue  be  kept  in  view  until  resolved  by  painting 
protective  coatings  on  the  piping.  It  was  detennined  that  the  pipe  manufacturer 
test  coupons  as  well  as  the  mill  test  certificate  American  Petroleum  Institute  (API) 
listing  showed  that  the  pipe  supplied  exceeded  Grade  B  pipe  strength  values. 

Parsons  conducted  an  independent  calculation  of  the  piping  thickness 
required.  This  analysis  was  based  on  the  requirements  as  outlined  in  ANSI  B31, 
31.3  as  applicable  for  3,200  psi  and  a  corrosion  value  of  .02  inches  which 
translates  to  4  or  6.67  microns  per  year  for  a  5-  and  3-year  project  life 
respectively.  It  is  the  Parsons  team’s  opinion  that  this  rate  of  corrosion  is 
consistent  with  the  industry  standards  used  to  predict  corrosion  rates  given  the 
existing  exposure.  Nevertheless,  it  is  also  the  Parsons  team’s  recommendation 
that  the  pipes  receive  appropriate  protective  coatings  to  ensure  that  corrosion  is 
controlled  throughout  the  life  of  the  temporary  pump  station. 

•  Unsuitable  pole-mounted  light  fixtures:  The  pole -mounted  light  fixtures  along 
the  bridges  and  walkways  designated  Type  F6  in  the  construction  drawings  are 
not  designed  for  hurricane  force  conditions;  therefore,  they  could  fall  and  destroy 
critical  components  of  the  pump. 

3.5  Operational  Vulnerabilities 
3.5.1  Findings 

This  section  identifies  parts  of  the  operation  and  process  that  are  vulnerable  to  the 

successful  operation  of  the  temporary  pump  stations  and  their  support  systems. 


Table  3-4 — Operational  Vulnerabilities 


Source  or  Cause  of 
Potential  Vulnerability 

Affected  Pump  Stations 

Assessed  Categories 

17th  Street 

Orleans 

London 

Criticality 

Vulnerability 

No  Alarms  on  the  hydraulic 
drives  for  over  pressure 

X 

X 

X 

3 

C 

Orleans  Avenue  Canal  pump 
station  sump  design 

X 

2 

C 

17th  Street  Canal  pump 
station  sump  design 

X 

2 

b 

Criticality:  1  -  High;  2  -  Medium;  and  3  -  Low 
Vulnerability:  a  -  Extreme;  b  -  Moderate;  and  c  -  Slight 
Note:  Refer  to  Tables  3-1  and  3-2  for  more  information. 
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•  No  alarms  on  the  hydraulic  drives  for  over  pressure:  The  team  observed  that 
there  were  no  hydraulic  oil  over-pressure  alarms.  There  are  pressure  safety  valves 
in  place  to  relieve  the  hydraulic  oil  over  pressure  but  there  are  no  alarms  or  visual 
indication  when  this  safety  valve  is  activated.  This  issue  is  rated  as  a  low  priority 
item  (3c)  and  appears  to  be  acceptable  to  the  agency. 

•  Orleans  Avenue  Canal  Pump  Station  Sump  Design:  No  physical  model  study 
was  conducted  of  the  Orleans  Avenue  Canal’s  pump  station  sump.  ERDC’s 
opinion  was  that  the  modifications  developed  at  London  Avenue  would  be 
equally  effective  at  Orleans  Avenue.  The  modifications  developed  at  London 
Avenue  were  installed  at  Orleans  Avenue.  Although  the  station  should  have  been 
modeled  to  confirm  the  design  that  was  developed  at  London  Avenue,  the 
potential  abnormalities  that  can  occur  are  likely  to  be  a  maintenance  issue. 
However,  since  the  station  was  not  modeled  to  confirm  the  design,  the 
vulnerability  for  this  station  is  2C. 

•  17th  Street  Canal  Pump  Station  Sump  Design:  Due  to  the  accepted 
performance  standards  (see  ANSI/HI  standards  criteria  for  “infrequent  pump 
operating  conditions”  as  stated  in  the  ERDC  January  2008  Report)  for  the  model 
sump  test  of  the  approach  flow  to  the  pump  station,  there  is  the  potential  for 
accelerated  wear  on  the  pump  impeller  and  bearings.  However,  at  this  station  the 
modifications  recommended  by  ERDC  were  not  implemented.  Not  constructing 
and  installing  the  recommended  modifications  might  result  in  performance  issues, 
such  as  cavitation  damage,  vibration,  and  bearing  wear.  These  abnormalities  will 
likely  develop  over  time.  Since  the  pumps  have  been  operated  during  the  last  two 
hurricanes,  it  is  recommended  that  a  pump  from  each  side  of  the  canal  be  checked 
and  assessed  for  damage  to  the  impeller  and  bearings  operations  and  maintenance 
as  part  of  an  ongoing  maintenance  plan.  If  abnormalities  arise,  the  design 
modifications  to  the  sump  should  be  implemented  to  improve  flow  to  the  pumps. 
The  vulnerability  for  this  station  is  2B. 
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3.6  Maintenance  Vulnerabilities 
3.6.1  Findings 

This  section  identifies  maintenance  components  and  processes  of  the  temporary  pump 
stations  and  their  support  systems  that  will  enhance  successful  performance  of  the 
stations. 


Table  3-5 — Maintenance  Vulnerabilities 


Source  or  Cause  of  Potential 
Vulnerability 

Affected  Pump  Stations 

Assessed  Categories 

17th  Street 

Orleans 

London 

Criticality 

Vulnerability 

Viscosity  change  and  contamination 
of  hydraulic  oil 

X 

X 

X 

3 

C 

Corrosion  of  hydraulic  pipe  fittings 
and  structure  that  are  sumerged 

X 

X 

X 

3 

C 

Degradation  of  impeller  due  to 
vortices 

X 

X 

2 

C 

Degradation  of  impeller  due  to 
vortices 

X 

2 

b 

Erosion  of  canal  bottom  &  wall  due 
to  high  water  velocities  at  pump 
intake  &  discharge 

X 

X 

X 

3 

c 

Criticality:  1  -  High;  2  -  Medium;  and  3  -  Low 

Vulnerability:  a  -  Extreme;  b  -  Moderate;  and  c  -  Slight 

Note:  Refer  to  Tables  3-1  and  3-2  for  more  information. 

•  Viscosity  change  and  contamination  of  hydraulic  oil:  Over  time  the  viscosity 
of  the  hydraulic  oil  will  change  due  to  pump  running  and  age  of  the  oil.  This  is 
rated  a  low  priority;  however,  there  should  be  a  maintenance  routine  included  in 
the  maintenance  program. 

•  Corrosion  of  hydraulic  pipe  fittings  and  parts  of  the  structure  that  are 
submerged:  Since  the  pipe  fittings  and  the  structure  that  support  the  pumps  are 
of  different  materials,  bi-metallic  corrosion  is  inevitable.  The  corrosion  of  these 
components  will  cause  failures  over  time  and  routine  subaqueous  checks  should 
be  included  in  the  maintenance  program. 

•  Degradation  of  impeller  due  to  vortices:  Vortices  can  cause  premature  wear  of 
the  impeller  and  bearings  due  to  cavitation.  Minimizing  pumping  operations  at 
levels  where  vortices  will  occur  is  recommended.  Consideration  should  be  given 
to  the  cost  versus  benefit  of  conducting  a  subsurface  monitoring  program  to 
ensure  that  excessive  wear  of  the  impellers  has  not  occurred.  Additionally,  close 
monitoring  of  the  pumps  during  operation  to  determine  if  vibrations,  not 
previously  observed,  begin  to  occur.  Should  unusual  vibrations  occur,  the 
removal  of  pumps  for  inspection  and  bearing  evaluation  may  be  necessary. 

•  Erosion  of  canal  structure  due  to  high  velocities:  Erosion  of  canal  structure 
may  occur  during  actual  high  discharge  events.  A  periodic  subsurface  monitoring 
system  will  mitigate  this  risk.  These  subsurface  monitoring  activities  should  be 
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conducted  in  such  a  way  as  to  allow  for  restorative  activities  before  the  wet 
season. 

3.7  Pump  Capacity  Analysis 

The  design  rainfall  event  serves  as  the  basis  of  this  analysis  to  detennine  the  required  and 
calculated  pumping  capacity  for  the  17th  Street,  London  Avenue,  and  Orleans  Avenue 
temporary  pump  stations.  For  the  purpose  of  this  report,  this  event  is  noted  as  the  design 
rainfall  event.  These  temporary  pump  stations  have  each  been  constructed  relatively  near 
the  outlet  of  each  canal  into  Lake  Pontchartrain,  and  upstream  of  a  new  temporary  gate 
structure.  Existing  SWB  pump  stations  pump  stonnwater  from  urban  drainage  areas  into 
the  respective  canal  that  serves  the  area.  During  Hurricane  Katrina,  the  water  level  in 
Lake  Pontchartrain  rose  due  to  storm  surge  and  the  effective  pumping  capacity  of  the 
SWB  pump  stations  is  decreased.  The  floodgates  near  the  end  of  each  of  the  canals  are 
intended  to  protect  the  canals  from  excessive  water  stage  levels  while  the  temporary 
pumping  stations  are  intended  to  pump  the  water  from  the  drainage  canals  to  Lake 
Pontchartrain.  This  arrangement  maintains  safe  canal  water  levels  and  avoids  failure  of 
the  protective  levees  on  either  side  of  the  canals.  These  safe  water  levels  were 
established  by  the  USAGE  geotechnical  engineers  and  are  unique  to  each  canal.  This 
analysis  is  based  on  the  maintenance  of  these  safe  water  levels  during  the  design  rainfall 
event. 

Simulated  hydrographs  were  developed  by  USACE  for  each  pump  station  indicating  that 
the  peak  flow  rates  for  the  design  rain  event  do  not  exceed  6  hours.  Table  3-6  shows  the 
required  pumping  capacities  for  each  temporary  pump  station  were  extrapolated  using  the 
provided  hydrographs  for  each  canal. 


Table  3-6 — 10-Year  Peak  Discharges  Extrapolated  from  Hydrographs 


Canal  Pump  Station 

10-Year  Peak  Discharge 
(cfs) 

10-Year  Peak  Discharge 
(cfs  rounded) 

17th  Street 

7,673 

7,700 

London  Avenue 

4,961 

5,000 

Orleans  Avenue 

1,883 

1,900 

Pumping  capacities  at  each  temporary  canal  pump  station  were  determined  using  the 
following  assumptions: 


•  Perfonnance  is  based  on  the  prototype  water  pump  curve  corrected  to  288  rpm  at 
the  Rineer  water  pump. 

•  Field  operating  friction  losses  in  the  pump  discharge  and  manifold  piping  using 
the  Hazen- Williams  friction  factor  C  equals  140  and  varying  K  factors  to  establish 
losses  due  to  elbow  friction,  entrance  losses,  and  manifold  friction  losses. 

•  The  approximate  maximum  flow  anticipated  from  the  hydraulic  drive  pumps  at 
each  location  is  based  on  operation  at  the  canal  safe  water  level  that  provides  the 
lowest  static  head. 

Installed  pumping  capacities  of  the  hydraulic  pumps  at  each  temporary  canal  pump 
stations  and  their  respective  installed  conditions  are  shown  below  in  Table  3-7. 


_ Final  Report _ 

PARSONS  3-8  February  27, 2009 


Independent  Engineering  Assessment  of  the 
New  Orleans  Temporary  Outfall  Canal  Pumps 


Vulnerability  Analysis 


Table  3-7 — Installed  Hydraulic  Drive  Pumping  Capacities 


Location 

Orleans 

Avenue 

London 

Avenue 

17th  Street 

Safe  Water  Level  Elevation  (ft) 

8 

5 

6 

Top  of  Discharge  Pipe  Elevation 

11 

11 

12 

Static  Head 

3 

6 

6 

Total  Number  of  Hydraulic  Pumps 

10 

12 

18 

Calculated  Discharge  per  Pump  (cfs) 

220 

216 

215 

Total  Calculated  Capacity  (cfs) 

2,201 

2,594 

3,864 

Although  analysis  of  the  direct  drive  pumping  units  at  both  the  17th  Street  Canal  and 
London  Avenue  Canal  and  the  portable  pumping  units  at  the  17th  Street  Canal  are  outside 
the  scope  of  this  report,  the  stated  capacity  of  these  additional  pumps  is  as  shown  in 
Table  3-8. 


Table  3-8 — Total  Stated  Capacity  of  Additional  Pumps 


Pumps 

17th  Street (cfs) 

London  Avenue  (cfs) 

Direct 

4,000 

2500 

Portable 

1400 

Total 

5400 

2500 

Therefore,  based  on  the  above  listed  and  calculated  capacity  of  the  hydraulic  pumps  and 
the  provided  capacity  of  the  direct  drive  and  portable  pumps,  the  capacity  of  the  pumping 
stations  has  been  determined  to  meet  or  exceed  the  required  pumping  capacity  as 
indicated  during  the  design  rainfall  event  and  is  summarized  in  Table  3-9. 


Table  3-9 — Total  Calculated  Pump  Station  Capacity  vs.  Total  Required 


Items 

17th  Street (cfs) 

London  Avenue 
(cfs) 

Orleans  Avenue 
(cfs) 

Hydraulic  Pumps 

3,864 

2,594 

2,201 

Direct  Drive  Pumps 

4,000 

2,500 

0 

Portable  Pumps 

1,400 

0 

0 

Total  Capacity 

9,264 

5,094 

2,201 

Total  Required 

7,673 

4,961 

1,883 
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3.8.1  Mechanical 

The  hydraulic  pump  system  consists  of  the  diesel  engine  drive  unit,  the  Denison 
hydraulic  pump,  the  hydraulic  pipe  system,  and  the  Rineer  hydraulic  water  pump.  The 
following  is  a  discussion  regarding  each  of  these  principle  elements: 

Diesel  Engines:  The  diesel  engines  powering  the  hydraulic  drive  pumps,  as 
manufactured  by  Caterpillar,  are  adequate  for  the  specified  operating  conditions.  It  is  the 
judgment  of  the  Parsons  team  that  these  engines  and  identical  models  have  been 
adequately  tested  to  perform  reliably  in  the  service  intended.  The  operating  speed  of 
these  engines  at  1800  rpm  is  consistent  with  current  industry  standards  for  diesel  engines 
serving  the  water  resource  industry;  therefore,  it  is  a  system  with  a  high  precedent. 

Denison  Hydraulic  Pump:  In  general,  pump  hydraulic  drive  systems  (including  the 
hydraulic  motors  mounted  on  the  pumps  and  the  hydraulic  pumps  mounted  on  the  diesel 
engines)  have  been  sized  to  operate  at  or  near  the  limits  of  their  capabilities  to  produce 
torque  and  horsepower  for  pump  operation  under  specified  conditions.  Field  test  results 
indicate  that  horsepower  and  torque  capacity  are  slightly  less  than  specified  and  than 
indicated  in  factory  tests.  However,  those  results  are  within  tolerances  allowed  for  field 
testing  by  the  ANSI/HI.  To  avoid  excessive  maintenance  and  premature  failure  of  the 
hydraulic  drive  systems,  the  hydraulic  motor  manufacturer  has  recommended  a  nonnal 
operating  pressure  of  3,200  pounds  per  square  inch  gauge  (psig)  during  normal  and 
continuous  operation.  To  date,  the  hydraulic  drive  systems  have  performed  as  required 
and  without  abnormal  maintenance  or  failures  during  Hurricane  Gustav  and  other  storm 
events. 

During  the  site  visit  and  through  interviews  with  the  USAGE  maintenance  organization, 
it  was  discovered  that  the  Aeroquip  fitting  on  the  Denison  pump  is  experiencing 
intennittent  leakage,  in  the  form  of  a  fine  mist.  It  is  our  understanding  that  this  leakage  is 
due  to  a  flange  interface  detail  that  is  currently  being  addressed  between  USACE  and  the 
manufacturer.  Resolution  to  this  issue  is  encouraged. 

Hydraulic  Pipe  System:  The  existing  piping  system  that  was  used  to  deliver  pressurized 
hydraulic  oil  to  the  Rineer  hydraulic  water  pump  was  reviewed  for  adequacy.  This 
analysis  resulted  in  a  confirmation  that  the  pipe  wall  thickness  meets  or  exceeds  the 
requirements  as  stated  in  the  ASME,  B31,  31.3  as  applicable  for  3,200  psi  hydraulic  oil. 
The  piping  system  as  witnessed  during  the  site  investigations  was  unprotected  from 
corrosion.  It  is  our  understanding  that  an  effort  is  currently  in  place  to  coat  the  piping  in 
order  to  avoid  excessive  corrosion. 

Rineer  Hydraulic  Motor:  This  unit  is  used  to  drive  the  pump  and  receives  its  thrust  via 
the  hydraulic  oil  delivered  from  the  Denison  hydraulic  pump.  As  discussed  in  Section  2, 
Testing  Adequacy  Analysis,  the  Rineer  motor  experienced  a  pulsing  effect  during  the 
acceptance  testing  period.  The  replacement  of  the  springs  appears  to  have  corrected  this 
anomaly  and  the  system  has  perfonned  satisfactorily  since. 
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3.8.2  Electrical 

The  electrical  distribution  (for  all  three  pump  stations,  the  standby  generators,  and  the 
associated  switches  that  control  the  distribution  for  the  hydraulic  pumps)  were  checked 
for  reliability  and  compliance.  The  pump  stations  are  powered  from  the  local  utility 
(Entergy)  distribution  100  kVA  transformers,  with  a  secondary  voltage  of  240  volts, 
single  phase,  60  Hertz  (Hz)  on  the  east  hank  and  west  bank  of  the  canals.  The  electrical 
construction  drawings  were  reviewed  and  electrical  load  calculations  perfonned  indicated 
the  following:  the  electrical  load  for  the  17th  Street  Canal  temporary  pump  station  was  88 
kVA;  the  Orleans  Avenue  Canal  temporary  pump  station  was  73  kVA;  and  the  London 
Avenue  Canal  temporary  pump  station  was  7 1  kVA,  on  each  side  of  the  canal  as  shown 
in  electrical  service  calculations. 

Switches,  panels,  and  generator  sizes  for  all  the  stations  were  verified  and  found  that  the 
fused  main  disconnects,  panels,  automatic  transfer  switches,  and  service  and  branch 
cables  were  correctly  sized  for  the  full  load  of  the  pump  stations.  Calculations  showed 
that  the  backup  generator  was  correctly  sized  at  100  kVA  for  loads  of  88  kVA,  73  kVA, 
and  71  kVA  per  side  of  the  pump  stations.  In  each  station,  there  were  additional  100  kVA 
generators  configured  to  take  over  the  backup  generators  in  event  of  a  backup  generator 
failure. 

The  electrical  lighting  was  examined,  and  it  was  noted  that  the  lighting  fixtures  over  the 
engine  platfonns  were  of  the  vapor  proof  fluorescent  type  fixtures.  These  fixtures  are  for 
wet  areas,  but  they  are  not  designed  for  hurricane  force  conditions.  Therefore,  they  could 
potentially  fall  and  destroy  critical  components  of  the  pump. 

The  pole  mounted  light  fixtures  along  the  bridges  and  walkways  designated  Type  F6  in 
the  construction  drawings  are  not  designed  for  hurricane  force  conditions.  Therefore,  they 
could  become  flying  debris. 

The  instrumentation  and  control  systems  were  checked.  The  system  uses  a  local  area 
network  architecture  linking  all  the  local  control  panels  to  a  supervisory  control  and  data 
acquisition  (SCAD A)  system  with  multimode  fiber  optic  cables.  This  architecture  allows 
communication  between  control  panels  and  the  SCADA  system  using  the  Modbus 
messaging  protocol.  The  network  is  a  ring  network  topology  allowing  for  media 
redundancy.  All  hydraulic  pumping  systems  are  equipped  with  local  controls  that  are 
autonomous  and  do  not  entirely  rely  on  the  SCADA  system  for  control.  Therefore,  they 
provide  redundancy  to  the  entire  control  system. 

The  US  ACE  temporary  pump  stations  are  connected  to  the  SWB  pump  stations  upstream 
of  the  canals  via  fiber  optic  cables.  This  allows  bidirectional  control  of  the  USAGE 
temporary  pump  stations  from  either  the  SWB  pump  stations  or  the  USAGE  temporary 
pump  stations.  The  communication  system  is  further  supported  with  redundancy  using  a 
microwave  radio  communication  local  area  network  transmitting  at  1 1  gigahertz  (GHz). 
This  wireless  medium  provides  transmission  of  SCADA  functions  over  Ethernet,  voice 
over  Internet  Protocol  (IP),  and  video  surveillance  to  observe  functionality  of  pumps  and 
gates.  This  arrangement  provides  an  effective  multipoint  to  point  transmission  system  to 
communicate  between  the  temporary  outflow  pump  stations  and  SWB  stations  using 
Ethernet  over  the  wireless  system.  For  further  redundancy  in  voice  communications,  the 
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temporary  pump  stations  operational  staff  is  equipped  with  satellite  phones  and  police 
band  radio  sets  as  well,  totaling  to  4  different  methods  of  voice  communications. 

3.8.3  Structural 

A  limited  evaluation  of  the  existing  structural  systems  was  performed  at  17th  Street, 
London  Avenue,  and  Orleans  Avenue  Canal  outfalls.  The  evaluation  is  based  on  onsite 
observation  for  condition  assessment  and  cursory  analysis  of  the  primary  landside 
support  systems.  The  systems  observed  and  their  associated  evaluations  are  as  follows: 

Landside  Support  Structure  and  Building:  The  supporting  structures  for  the  pumping 
systems  consist  of  steel  pipe  piles  at  17th  Street  and  H-piles  at  both  Orleans  Avenue  and 
London  Avenue.  Visual  observation  of  these  structures  indicates  that  their  condition  is 
satisfactory  although  no  protective  coatings  were  used.  A  cursory  analysis  indicates  that 
the  gravitational  capacity  of  the  floor  system  meets  or  exceeds  the  requirements  of  the 
ASCE  7-02  recommendations.  ASCE  7-02  is  the  applicable  standard  for  this  installation 
which  provides  minimum  design  loads  for  buildings  and  other  structures.  The  wind  loads 
used  for  the  pre-engineered  building  were  also  evaluated.  Again,  the  loads  used  met  or 
exceeded  ASCE  7-02  recommendations. 

Control  Room:  During  emergency  response  events,  the  operators  occupy  the  control 
room  at  each  of  the  subject  canal  pumping  stations.  The  room  is  constructed  of  thick 
concrete  walls,  heavy  steel  doors,  and  the  air  conditioning  unit  that  is  hardened  with  steel 
framing  providing  protection  from  wind  pressures  and  flying  debris.  These  evidential 
observations  indicate  that  the  control  room  meets  or  exceeds  the  minimum  requirements 
for  wind  protection. 

Erosion  Protection:  A  cursory  review  of  the  general  configuration  and  activities 
conducted  relating  to  the  erosion  protection  before  and  after  the  flood  wall  was  included 
in  this  review.  The  erosion  protection  consists  of  rip-rap  protection  ranging  from  2  to  4 
feet  in  thickness  over  a  soil  cement  substrate.  It  is  the  Parsons  team’s  opinion  that  due 
diligence  was  exercised  in  this  effort. 

Windborne  Debris:  The  landside  station  hydraulic  drive  systems  are  somewhat 
protected  by  a  chain  link  fence  that  surrounds  the  platfonn.  The  waterside  hydraulic 
drive  systems  at  the  17th  Street  Station  are  unprotected.  Given  the  excess  capacity  at  the 
17th  Street  station  this  exposure  risk  is  mitigated.  The  chain  link  protection  is 
unsupported  between  the  column  supports.  Additional  support  for  the  chain  link  fence 
would  improve  its  flying  debris  protective  benefit. 

3.8.4  Operational 

The  operational  procedures  for  the  temporary  pump  systems  were  reviewed  with  the  HPO. 
The  operational  procedures  include  continuous  communication  with  the  New  Orleans 
SWB  representative  at  the  upstream  pump  station.  Communications,  as  described  in  the 
Electrical  Section  of  this  report,  includes  redundant  systems  including  fiber-optic  line 
connections,  radio,  microwave  and  out-of-state  cell  phones. 

A  review  of  the  Operations  Instruction  Manual  (OIM)  indicates  that  the  procedures  in 
place  meet  expectations  for  pre-storm,  stonn  and  post-storm  activities.  Substantially,  the 
OIM  provides  detailed  descriptions  for  the  following: 
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•  Safe  canal  levels 

•  Gate  closure 

•  Pumping  activities 

•  Organizational  and  chain  of  command  structure 

•  Fuel  supply 

•  Electrical  power 

•  Water  level  controls 

•  Equipment  failure  and  diagnostic  procedures 

•  Reporting  and  logging  protocol 

It  is  the  Parsons  team’s  opinion  that  the  collaboration  between  the  HPO  and  the  SWB  is 
essential  to  successful  operations  of  the  flood  control  system.  Substantially,  the 
cooperative  operation  of  the  two  pumping  systems  tends  to  create  a  condition  whereby 
the  rise  in  canal  levels  between  the  stations  reduces  overall  pumping  capacity  of  the  SWB 
due  to  the  increased  tailwater  while  increasing  the  capacity  of  the  Temporary  Pump 
Stations  due  to  the  reduction  in  total  dynamic  head.  Historical  data  regarding  the  Gustav 
and  Ike  events  indicates  that  the  operational  plan  was  successful. 

3.8.5  Maintenance 

The  maintenance  plan  was  reviewed  using  the  check  list  provided  by  the  HPO  to  the 
Parsons  team.  The  plan  indicates  elements  of  the  system  that  are  checked  and  the 
frequency  of  those  verifications.  The  plan  includes  checks  of  the  following  elements  of 
the  system: 

•  Engine  before  start 

•  Hydraulic  system 

•  Engine  pre-start 

•  Hydraulic  system  after  start  before  engaging 

•  Hydraulic  system  after  start  and  after  engaging 

•  SCADA  system 

This  review  indicates  that  the  primary  pump  system  components  are  subject  to 
verification  and  a  log  of  the  operating  record  is  maintained  to  document  performance. 
The  perfonnance  logs  for  all  hydraulic  pumping  systems  including  their  support  systems 
were  provided  by  HPO.  This  performance  log  indicates  that  only  minor  corrections  have 
been  required  since  the  completion  of  the  installation  acceptance  testing. 

The  maintenance  program  is  sufficient  for  its  intended  purpose  with  regard  to  the 
machinery  elements.  In  order  to  maintain,  reliability  of  diesel  drives  and  hydraulic 
pumping  units,  periodic  operation  of  the  system  is  recommended  by  the  manufacturer. 
The  Parsons  team  understands  that  the  systems  are  started  every  2  weeks  during  the 
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hurricane  season  and  once  per  month  during  the  off  season.  This  is  considered 
satisfactory  for  maintaining  the  reliability  of  the  system. 

There  is  no  indication  that  a  subsurface  monitoring  program  is  currently  in  place. 
Pumping  systems,  especially  those  in  salt  water  or  brackish  water  conditions,  should  be 
checked  periodically  to  verify  that  excessive  corrosion  has  not  occurred.  Additionally, 
the  erosion  protection  system  (rip-rap  upstream  and  downstream)  cannot  be  verified 
without  the  benefit  of  subsurface  monitoring.  Finally,  as  indicated  in  the  Section  2, 
Testing  Adequacy  Analysis  of  this  report,  periodic  visual  inspection  of  the  pump 
impellers  to  verify  that  excessive  wear  has  not  occurred  as  a  function  of  the  potential 
vortices  is  not  included  in  the  maintenance  program.  The  Parsons  team  suggests  that 
subsurface  monitoring  could  be  useful  to  the  maintenance  program.  Typically,  this  could 
be  conducted  before  each  hurricane  season  and  in  time  to  enact  mitigation  actions,  if 
necessary. 

The  current  logging  effort  is  conducted  using  a  local  data  base  and  the  forms  as  prepared 
by  HPO.  We  understand  that  the  HPO  is  currently  preparing  to  participate  in  a  computer 
based  maintenance  management  system  called  the  USAGE  Facilities  &  Equipment 
Maintenance  System  (FEM)  that  will  allow  for  tracking  trends  associated  with  the 
reporting  of  maintenance  and  operational  activities.  It  is  the  Parsons  team’s  opinion  that 
the  use  of  this  program,  properly  applied,  will  increase  the  reliability  and  predictability  of 
the  system. 

No  significant  vulnerabilities  were  identified  with  regard  to  the  existing  maintenance 
program. 

3.9  Conclusions 
3.9.1  Mechanical 

The  Parsons  team  conclusions  for  mechanical  systems  vulnerability  are  as  follows: 

•  The  design  rainfall  event  used  to  size  the  temporary  canal  pump  stations  has  peak 
flow  duration  on  the  order  of  4  to  6  hours  and  this  criterion  limits  substantially  the 
duration  needed  for  the  peak  pumping  rate. 

•  Installed  pumping  capacity  at  the  17th  Street,  the  Orleans  Avenue,  and  London 
Avenue  pumping  station  meets  or  exceeds  the  service  intended. 

•  The  diesel  engines  are  appropriately  sized  and  are  suitable  for  their  intended  use. 

•  The  Denison  hydraulic  pump  and  motor,  despite  earlier  issues  with  the  cam  wear  and 
selection,  are  sized  correctly  and  are  demonstrating  a  high  level  of  reliability 
evidenced  by  the  recent  performance  during  the  Gustav  and  Ike  storm  events. 

•  The  applicable  standard  for  the  power  transmission  piping  (hydraulic  oil)  is  the 
ASME  B31,  31.3.  Pipe  thickness  calculations  were  completed  based  on  the  guidance 
within  B31,  31.3  as  was  appropriate  and  sufficient. 

•  The  Rineer  motor  driving  the  water  pump  experienced  the  pulsing  conditions  during 
the  acceptance  testing  period.  The  replacement  of  the  spring  system  appears  to  have 
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rectified  that  issue  and  their  performance  appears  to  be  suitable  for  their  intended 
purpose. 

3.9.2  Electrical 

Parsons  team’s  conclusions  for  electrical  systems  vulnerability  are  as  follows: 

•  It  is  the  Parsons  team’s  opinion  the  electrical  distribution  and  double  backup  systems 
are  capable  of  full  power  and  standby  power  distribution  for  the  hydraulic  pumps, 
which  fully  complies  with  the  National  Fire  Protection  Association  (NFPA)  Code  70 
for  all  three  stations. 

•  The  vulnerability  of  the  light  fixtures  is  not  detrimental  to  the  operation  of  the 
temporary  pump  station.  The  asset  risk  level,  for  both  the  fluorescent  fixtures  and  the 
pole  mounted  fixtures  are  detennined  at  Level  3c. 

•  It  is  the  Parsons  team’s  opinion  that  there  is  enough  redundancy  in  the 
communications  system  for  effective,  reliable  communication,  and  control  of  the 
interim  pump  stations  during  an  emergency  response. 

3.9.3  Structural 

Parsons  team’s  conclusions  for  structure  and  building  vulnerability  are  as  follows: 

•  The  landside  support  structure  and  building  were  reviewed  for  capacity  and  compared 
to  the  requirements  of  ASCE  7.  This  evaluation  indicates  that  the  structures  meet  or 
exceed  the  minimum  requirements  for  design  loads  as  prescribed  by  ASCE  7. 

•  The  Control  Room  was  observed  during  a  site  visit  and  the  construction  is  consistent 
with  the  requirements  for  wind  load  resistance  as  required  by  ASCE  7. 

•  The  erosion  protection  system  for  the  flood  gates  is  consistent  with  industry  standards 
for  structures  of  this  type. 

•  There  are  no  apparent  vulnerabilities  associated  with  the  structural  system  as 
designed  and  constructed. 

3.9.4  Operational 

Parsons  team’s  conclusions  for  operational  vulnerability  are  as  follows: 

•  The  operations  requirements  are  outlined  in  the  OIM.  This  manual  indicates  due 
diligence  was  exercised  in  its  creation. 

•  The  operations  protocol  includes  close  coordination  between  USAGE  and  the  SWB 
during  stonn  events. 

•  This,  coupled  with  redundant  communications  systems  indicates  that  no  apparent 
vulnerabilities  are  associated  with  the  operational  plan  and  implementation  program. 

3.9.5  Maintenance 

Parsons  team’s  conclusions  for  maintenance  vulnerability  are  as  follows: 

•  The  maintenance  program  is  outlined  in  the  form  of  a  checklist  for  the  mechanics  to 
follow  during  routine  maintenance  of  the  pumping  systems.  This  outline  appears  to 
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satisfactorily  meet  the  objective  of  proper  mechanical  maintenance  and  is  considered 
satisfactory  for  maintaining  the  reliability  of  the  system. 

•  The  plan  does  not  indicate  a  subsurface  monitoring  system  that  is  considered 
beneficial  to  verifying  that  the  structures  below  the  water  surface  are  not  exhibiting 
excessive  corrosion,  excessive  degradation  of  pump  components  and  is  also 
considered  beneficial  to  verifying  that  the  erosion  protection  system  has  not  degraded. 

3.10  2008  Performance  during  Hurricanes  Gustav  and  Ike 

The  Parsons  team  reviewed  the  Temporary  Outflow  Canal  Pumps  performance  reports 
for  the  two  hurricanes  Gustav  and  Ike  and  gathered  the  following  infonnation: 

3.10.1  Hurricane  Gustav 

On  August  31,  2008,  Hurricane  Gustav  made  landfall,  but  New  Orleans  escaped  the  full 
force  of  the  hurricane  as  the  storm  entered  near  the  town  of  Cocodrie,  Louisiana.  The 
hurricane  was  a  Category  2  storm,  but  the  Louisiana  coast  experienced  torrential  rain  and 
high  winds  of  approximately  100  mph,  which  generated  a  stonn  surge  in  Lake 
Pontchartrain. 

The  USACE  team  received  orders  the  next  day  to  close  the  canal  gates  at  the  temporary 
outflow  canal  pump  stations,  cutting  off  the  canals  outflow  to  Lake  Pontchartrain  in 
anticipation  of  the  storm  surge  associated  with  the  high  winds. 

The  records  from  USACE  show  that  at  the  London  Avenue  Canal’s  temporary  pump 
station  all  of  the  hydraulic  and  direct  drive  pumps  were  put  into  service.  The  total  flow 
from  the  SWB  pumps  was  5,050  cfs  on  that  day.  With  both,  the  hydraulic  pumps  and  the 
direct  drive  pumps  running  at  the  London  Avenue  Canal  pump  station,  the  total  flow 
pumped  into  Lake  Pontchartrain  was  5200  cfs,  reducing  the  level  of  the  canal  down  to  a 
safe  level.  The  London  Avenue  Canal  temporary  pump  station  ran  for  27  minutes, 
matching  the  discharge  from  the  SWB  and  maintained  a  safe  water  level  in  the  London 
Avenue  Canal.  After  the  first  27  minutes,  4  hydraulic  pumps  were  shut  down  as  the  canal 
level  got  too  low  for  the  pumps  to  function  efficiently.  The  4  hydraulic  pumps  were 
started  again  after  48  minutes  and  all  pumps  ran  for  another  55  minutes  before  the  4 
hydraulic  pumps  were  shut  down  again  due  to  low  levels  in  the  canal.  The  hydraulic 
pumps  ran  for  3  days  intermittently  with  various  numbers  of  pumps  taken  in  and  out  of 
service  to  control  the  canal  level  with  the  gates  shut  for  36  hours.  Out  of  the  12  hydraulic 
pumps  in  service,  2  pumps  were  taken  out  of  service  due  to  minor  malfunctions.  The 
fault  on  one  was  a  disconnected  control  wire,  and  the  second  was  a  loose  flange  bolt. 
There  were  no  pump  mechanical  malfunctions.  These  pumping  activities  are  summarized 
in  Table  3-10. 

The  records  show  that  at  the  17th  Street  Canal  temporary  pump  station  and  all  of  the 
hydraulic  and  direct  drive  pumps,  except  the  portable  pumps,  were  put  into  service.  The 
total  flow  from  the  SWB  pumps  was  approximately  5,050  cfs.  With  all  available  pumps 
running  at  the  17th  Street  Canal’s  temporary  pump  station  (the  portable  pumps  were  not 
included),  the  total  flow  pumped  into  Lake  Pontchartrain  was  approximately  7,240  cfs. 
The  17th  Street  Canal  temporary  pump  station  ran  for  52  minutes,  matching  the  discharge 
from  the  SWB  and  kept  the  level  of  the  17th  Street  Canal  at  a  safe  level.  After  the  52 
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minutes,  all  direct  drive  pumps,  with  the  exception  of  2  pumps,  were  taken  out  of  service, 
leaving  all  hydraulic  pumps  running  for  1  hour.  As  the  canal  water  level  dropped,  the  2 
direct  drive  pumps  were  taken  out  of  service  leaving  all  hydraulic  pumps  running.  The 
hydraulic  pumps  ran  for  2  days  intermittently  with  various  numbers  of  pumps  taken  in 
and  out  of  service  to  control  the  canal  level  with  the  flood  gates  shut  for  a  total  of  18 
hours.  Out  of  the  18  hydraulic  pumps  in  service,  1  hydraulic  pump  was  taken  out  of 
service  due  to  a  hydraulic  leak  on  a  drive  unit.  There  were  no  other  pump  mechanical 
malfunctions.  These  pumping  activities  are  summarized  in  Table  3-11. 

3.10.2  Hurricane  Ike 

On  the  morning  of  September  12,  2008,  Hurricane  Ike  approached  landfall  as  a  Category 
3  wind  force  generating  a  stonn  surge  in  Lake  Pontchartrain  of  approximately  5.2  feet. 

The  records  show  that  the  USAGE  team  experienced  rain  and  winds  of  around  25  mph  at 
the  temporary  pump  stations,  when  they  received  orders  to  close  the  canal  gates  cutting 
off  the  canal’s  outflow  to  Lake  Pontchartrain. 

Records  show  that  the  west  bank  direct  drive  pumps  were  put  into  service  at  the  London 
Avenue  Canal  temporary  pump  station.  The  total  flow  from  the  SWB  pumps  was 
approximately  1,000  cfs,  with  the  pumps  running  at  the  London  Avenue  Canal  temporary 
pump  station;  the  total  flow  pumped  into  Lake  Pontchartrain  was  approximately  1,200  cfs. 
The  London  Avenue  Canal  temporary  pump  station  ran  for  58  minutes,  matching  the 
discharge  from  the  SWB  and  kept  the  level  of  the  London  Avenue  Canal  at  a  safe  level. 
After  the  first  58  minutes,  2  direct  drive  pumps  were  shut  down  as  the  canal  water  level 
dropped  too  low  for  the  pumps  to  function  efficiently.  An  hour  and  10  minutes  later,  6 
hydraulic  pumps  were  started  again  and  ran  for  another  35  minutes  before  the  hydraulic 
pumps  were  shutdown  again  as  the  canal  level  got  low.  The  hydraulic  pumps  ran  for  4 
days  intermittently  with  various  numbers  of  pumps  taken  in  and  out  of  service  to  control 
the  canal  level  with  the  gates  shut  for  60  hours.  There  were  no  pump  malfunctions 
during  this  hurricane  event,  as  can  be  seen  in  Table  3-12. 

Records  show  that  at  the  17th  Street  Canal  temporary  pump  station,  10  direct  drive 
pumps,  and  6  hydraulic  pumps  were  put  in  service.  The  total  flow  from  the  SWB  pumps 
was  4,500  cfs,  with  10  direct  drive  pumps  and  6  hydraulic  pumps  running  at  the  17th 
Street  Canal  temporary  pump  station,  the  total  flow  pumped  into  Lake  Pontchartrain  was 
5,200  cfs.  The  hydraulic  pumps  ran  for  2  days  intennittently  with  various  numbers  of 
pumps  taken  in  and  out  of  service  to  control  the  canal  level  with  the  gates  shut  for  36 
hours.  Out  of  the  18  hydraulic  pumps  in  service,  1  hydraulic  pump  was  taken  out  of 
service  due  to  a  hydraulic  hose  leak  on  a  drive  unit.  There  were  no  other  pump 
mechanical  malfunctions,  as  can  be  seen  in  Table  3-13. 

Records  show  that  while  the  stonn  surge  was  not  enough  to  close  the  gates  at  the  Orleans 
Avenue  Canal,  the  USAGE  canal  team  decided  to  take  advantage  of  the  water  levels  to 
exercise  the  pumps  and  10  hydraulic  pumps  were  put  in  service  at  the  Orleans  Avenue 
Canal  pump  station.  The  5  east  bank  pumps  were  put  in  service  first  and  ran  for  3  hours, 
the  total  flow  pumped  into  Lake  Pontchartrain  was  1,044  cfs.  After  3  hours,  the  pumps 
were  taken  out  of  service  and  the  west  bank  pumps  were  put  in  service  and  ran  for  3 
hours  at  1,054  cfs.  The  hydraulic  pumps  ran  for  two  periods  of  3  hours  each.  Total 
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amount  of  water  pumped  at  this  facility  was  over  22  million  gallons  in  this  period  without 
any  pump  major  malfunctions  with  the  10  pumps.  One  HPU  experienced  a  high 
temperature  oil  indication  that  turned  out  to  be  an  electrical  short  circuit.  This  was 
repaired  and  the  HPU  was  put  back  in  service.  These  pumping  activities  are  recorded  in 
Table  3-14. 

3.10.3  Conclusions 

Parsons  team’s  conclusions  area  as  follows: 

1 .  During  Hurricane  Gustav  the  following  occurred: 

•  Two  pumps  at  the  London  Avenue  Canal  temporary  pump  station  had  minor 
malfunctions,  one  a  loose  bolt  and  the  second,  a  loose  control  wire. 

•  A  direct  drive  pump  and  a  hydraulic  pump  at  the  17th  Street  Canal  temporary 
pump  station  had  minor  malfunctions;  the  direct  drive  experienced  a  loose 
control  wire,  and  the  hydraulic  pump  experienced  a  leak  in  the  hydraulic  drive 
unit  hose. 

2.  During  Hurricane  Ike  the  following  occurred: 

•  There  were  no  pump  malfunctions  at  the  London  Avenue  Canal  temporary 
pump  station  during  the  hurricane. 

•  One  hydraulic  pump  experienced  a  failure  in  the  hydraulic  hose  at  the  17th 
Street  Canal  temporary  pump  station. 

3.  Pump  Activities  during  Hurricane  Gustav: 

The  pump  activities  during  Hurricane  Gustav  are  summarized  in  Tables  3-10,  and 
3-11.  As  shown  in  Tables  3-10  and  3-11,  the  total  number  of  hydraulic  pump 
hours  for  London  Avenue  was  105.3  pump  hours  and  for  the  17th  Avenue  was 
224.5  pump  hours.  The  total  number  of  direct  drive  pump  hours  for  London 
Avenue  was  75.2  pump  hours  and  for  the  17th  Avenue  was  28  pump  hours.  As 
shown  in  both  pump  stations  values,  the  hydraulic  pumps  were  mainly  used  to 
bring  the  canal  levels  down.  It  took  3  hours  to  bring  the  canal  down  by  2  feet. 

4.  Pump  Activities  during  Hurricane  Ike: 

The  pump  activities  during  Hurricane  Ike  are  summarized  in  Tables  3-12  and  3-13. 
As  shown  in  Tables  3-12  and  3-13,  the  total  number  of  hydraulic  pump  hours  for 
London  Avenue  was  15.93  pump  hours  and  for  the  17th  Avenue  was  28.6  pump 
hours.  The  total  number  of  direct  drive  pump  hours  for  London  Avenue  was  60 
pump  hours  and  for  the  17th  Avenue  was  76.4  pump  hours.  As  shown  in  both 
pump  station  tables,  the  direct  drives  were  used  more  than  the  hydraulic  pumps  to 
bring  the  canal  levels  down. 

In  Hurricane  Gustav,  two  stations  pumped  a  total  of  more  than  1.7  billion  gallons  of 
water  (pump  rates  were  quoted  by  the  US  ACE)  into  Lake  Pontchartrain  with  two 
hydraulic  pumps  experiencing  minor  malfunctions.  In  Hurricane  Ike,  the  two  stations 
pumped  a  total  of  approximately  half  a  billion  gallons  of  water  (pump  rates  were  quoted 
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by  the  USAGE)  into  Lake  Pontchartrain  with  one  hydraulic  pump  experiencing  a 
malfunction. 

During  Hurricane  Ike  the  hydraulic  pumps  at  the  Orleans  Avenue  Canal  Pump  Station 
were  tested  for  perfonnance.  The  gates  were  not  shut  as  the  canal  was  at  a  safe  level  of 
5.2  feet.  The  pumps  were  tested  successfully  with  pumps  running  at  near  maximum 
output  with  one  minor  malfunction. 

Minor  pump  malfunctions  are  expected  for  a  large  group  of  pumps  such  as  this 
installation.  The  industry  MTBF  (mean  time  between  failures)  ANSI  standard  for  a 
single  axial  pump  continually  running  is  2.5  years  between  failures  in  the  U.S.  This 
means  it  is  expected  that  a  single  axial  pump  continuously  running  will  breakdown  within 
a  2.5-year  period.  With  hydraulic  pumps,  one  can  expect  a  much  lower  MTBF  value  as 
additional  drive  components  (besides  the  engine  prime  mover)  such  as  the  hydraulic  drive 
unit  adds  to  the  probability  of  failure.  It  can  be  expected  that  a  hydraulic-driven  pump 
will  have  a  lower  MTBF,  and  it  will  be  more  prone  to  failures  than  a  single  prime  mover 
pump.  However,  these  pumps  will  be  used  infrequently  versus  one  that  will  run 
continuously  for  2.5  years.  This  will,  therefore,  inevitably  prolong  the  life  of  the  pump  to 
a  much  greater  extent  than  a  standard  continuously  running  pump. 

HPO  Interim  Closure  Structure  (ICS)  Records  also  show  that  randomly  selected  pumps 
ran  at  near  the  recommended  3,200  psi  hydraulic  pressure,  proving  that  these  pumps  are 
capable  of  the  recommended  pressures.  Most  of  the  pumps  ran  at  an  average  of  80  to 
95%  of  the  recommended  pressure.  While  these  pumps  are  capable  of  the  recommended 
extents,  it  is  not  normal  practice  in  all  industries  to  run  machinery  at  maximum 
performance.  This  is  inefficient  and  causes  undue  mechanical  stress  to  any  mechanical 
system,  which  will  result  in  premature  failure  of  components. 

The  Parsons  team  is  satisfied  that  the  pumps  have  functioned  to  industry  standards,  and 
the  team  is  confident  that  the  pumps  will  perfonn  as  designed  and  constructed  to  cope 
with  the  design  rainfall  event. 
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Table  3-10 — London  Avenue  ICS  -  Hurricane  Gustav  Outfall  Canal  Closure  Activities 


|  LONDON  AVENUE  ICS  -  HURRICANE  GUSTAV  OUTI 

rALL  CA 

NAL  CLOSURE  ACTIVITIES  3 

Time 

Run  Time 
mins 

Hydraulic  Pumps 

DD  Drv  Pumps 

Total 
Hydraulic 
Pump  Row 
CFS 

Volume  Pumped 
Gallons  by 
Hydraulic  pumps 

Total  Direct 
Drive  Flow 
CFS 

Volume  Pumped 
Gallons  by  Direct 
Drive  pumps 

Canal 
Level  in 
Ft. 

Comments 

El 

E2 

E3 

E4 

E5 

e>: 

W 

1 

W 

2 

w 

3 

w 

4 

W 

5 

W 

6 

E7 

E8 

E9 

El 

0 

W 

7 

W 

8 

w 

9 

W 

10 

|September  1, 2008  j 

19:16 

19:25 

9 

• 

• 

• 

• 

• 

• 

• 

• 

0 

2,400 

9,694,080 

4.2 

19:25 

19:52 

27 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

2,800 

33,929,280 

2,400 

29,082,240 

4.2 

19:52 

20:40 

48 

• 

• 

• 

• 

• 

• 

• 

• 

0 

2,400 

51,701,760 

4.2 

20:40 

21:35 

55 

□□□□□□□□□□□□□□DDQDDC 

2,800 

69,115,200 

2,400 

59,241,600 

4.2 

21:35 

22:05 

30 

• 

•  • 

E 

□E 

B 

E 

E 

E 

B 

E 

1 

1 

1 

1 

E 

E 

• 

• 

480 

6,462,720 

1,200 

16,156,800 

4.2 

Reduced  Speed  of  Hydraulic  Pumps 

22:05 

0:00 

115 

• 

•  • 

• 

•  • 

• 

• 

• 

• 

• 

• 

1,800 

92,901 ,600 

0 

2.5 

|September2, 2008  j 

00:00 

01:40 

ICO 

• 

• 

• 

. 

• 

750 

33,660,000 

750 

33,660,000 

2.5 

01:40 

04:20 

160 

• 

• 

• 

• 

• 

500 

35,904,000 

0 

2.5 

07:10 

07:56 

46 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

1,200 

24,773,760 

0 

2.5 

07:56 

08:40 

44 

• 

• 

• 

• 

• 

• 

600 

1 1 ,848,320 

0 

2.5 

08:40 

10:10 

90 

• 

• 

• 

300 

12,117,600 

0 

2.5 

let  =  7.48  us  gal 

13:58 

15:15 

77 

• 

• 

• 

• 

• 

• 

• 

• 

0 

2,400 

82,938,240 

2.5 

16:15 

17:30 

75 

• 

• 

• 

• 

• 

• 

600 

20,196,000 

0 

2.5 

17:30 

18:35 

65 

• 

• 

• 

• 

• 

• 

600 

17,503,200 

0 

2.5 

18:35 

19:30 

55 

• 

• 

• 

• 

1,200 

29,620,800 

0 

2.5 

2w-off  line  due  to  leaking  hydraulic  oil  flange 

19:30 

20:20 

50 

• 

• 

• 

• 

• 

800 

17,952,000 

0 

2.5 

4w-off  line  due  to  high  gear  oil  temp.  ! 

22:50 

23:50 

60 

• 

• 

0 

600 

16,156,800 

2.5 

[September  3,  2008  j 

05:15 

16:10 

655 

• 

• 

• 

• 

0 

1,200 

352,756,800 

06:10 

06:20 

10 

• 

• 

0 

600 

2,692,800 

06:20 

0 

0 

Gates  Opened 

j  Total  Hours 

29.52 

co 

to 

CO 

co 

to 

to 

co 

to 

co 

o 

<b 

to 

to 

CM 

<0 

co 

co 

co 

<0 

co 

<0 

co 

CM 

0 

CM 

UO 

CM 

0 

cm 

Total  Gals. 

405,984,480 

Total  Gals. 

654,081,120 

1 

Final  Report 

3-20 


February  27,  2009 


Independent  Engineering  Assessment  of  the 
New  Orleans  Temporary  Outfall  Canal  Pumps 


Vulnerability  Analysis 


Table  3-11 — 17th  Street  ICS  -  Hurricane  Gustav  Outfall  Canal  Closure  Activities 


17th  STREET  ICS  -  HURRICANE  GUSTAV  OUTFALL  CANAL  CLOSURE  ACTIVITIES 
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Table  3-12 — London  Avenue  ICS  -  Hurricane  Ike  Outfall  Canal  Closure  Activities 
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Table  3-13 — 17th  Street  ICS  -  Hurricane  Ike  Outfall  Canal  Closure  Activities 


1 7th  STREET  ICS  -  HURRICANE  IKE  OUTFALL  CANAL  CLOSURE  ACTIVITIES 
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Table  3-14 — Orleans  Avenue  ICS  -  Hurricane  Ike  Outfall  Canal  Closure  Activities 


ORLEANS  AVENUE  ICS  -  HURRICANE  IKE  OUTFALL  CANAL  ACTIVITIES 


Time 

Run  Time 
mins 

H,'*aulc  Pump* 

Tolal 
Hydraulic 
Pump  Flow 

CFS 

Volume  Pumped 
Gallons  by  Hydraulic 
pumps 

Comments 

El 

E2 

E3 

E4 

E5 

W1 

W2 

W3 

W4 

W5 

September  12, 2038 

CUBIC  F 

EET 

PER 

SECOND 

14:30 

1445 

15 

208  2C6 

238 

210 

210 

1.044 

7.026,208 

14:45 

15:00 

15 

208  2M 

238 

210 

210 

1,044 

7.028,208 

15:00 

15:15 

15 

206 

2M 

208 

210 

210 

1044 

7,026.208 

15:15 

15:30 

15 

208 

2C6 

208 

210 

210 

1.044 

7.028,208 

E5  -  Oil  Temp  Sensor  Failure 

15:30 

15:45 

15 

208 

2« 

238 

210 

834 

5,614,488 

15:45 

16:00 

15 

208  2C6 

238 

210 

834 

5,614.488 

16:00 

16:15 

15 

208  2C6 

238 

210 

834 

5.614.488 

16:15 

16:30 

15 

208  2C« 

238 

210 

834 

5.614  488 

16:30 

1645 

15 

208  2C6 

208 

210 

834 

5.614  468 

16:45 

17:00 

15 

208  2C6 

238 

210 

834 

5,614  488 

17:00 

17:15 

15 

208  2C6 

238 

210 

834 

5.614  468 

17:15 

17:30 

15 

2, 

2C6 

208 

210 

834 

5.614.488 

17:30 

17:45 

15 

2- 

2C6 

238 

210 

834 

5.614  468 

18:00 

18:15 

15 

210  236 

208 

206 

220 

210 

1.264 

8.509.248 

E5  Restart  feat  after  repair 

18:15 

18:30 

15 

210  238 

208  206 

220 

210 

1.264 

8,509,248 

16:30 

18:45 

15 

210 

236  208 

206 

220 

210 

1.264 

8,509.248 

18:45 

19:00 

15 

210 

206 

208  206 

220 

210 

1.264 

8,509,248 

19:00 

19:15 

15 

210 

238 

208 

206 

220 

210 

1.264 

8,509,248 

19:15 

19:30 

15 

210 

236  208  2C6 

220 

210 

1.264 

8.509.248 

19:30 

1945 

15 

210 

236 

208 

206 

220 

210 

1.264 

8,509.248 

19:45 

20.00 

15 

210  236 

206 

206 

220 

210 

1.264 

8,509.248 

20:00 

20: 15 

15 

210 

238 

208 

206 

220 

210 

1.264 

8,509.248 

20:15 

20:30 

15 

210 

238 

208 

206 

220 

210 

1,264 

8,509,248 

2030 

2045 

15 

210 

236 

208  206 

220 

210 

1.264 

8,509  248 

20:45 

21:00 

15 

236 

208 

206 

220 

210 

1.054 

7.096.528 

21:00 

21:15 

15 

206 

206 

206 

220 

210 

1.054 

7,096.528 

21:15 

21:30 

15 

236 

206 

206 

220 

210 

1,054 

7.096,528 

21:30 

1 

0 

Total  Hcors 

3.50 

Total  Gals. 

114.888.312 
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